The Token Tax: Systematic Bias in Multilingual Tokenization
arxiv.org ↗(2025) - quantifies how tokenization fertility predicts model accuracy across languages, exposing structural cost penalties for morphologically complex and low-resource languages.
AboutVisit arxiv.org ↗
(2025) - quantifies how tokenization fertility predicts model accuracy across languages, exposing structural cost penalties for morphologically complex and low-resource languages.
This entry links to an external website rather than a GitHub repository, so there's no README to display here.