Reducing Tokenization Premiums for Low-Resource Languages
arxiv.org ↗(2026) - post-hoc vocabulary additions that coalesce multi-token character sequences for low-resource languages, reducing inference cost without retraining.
AboutVisit arxiv.org ↗
(2026) - post-hoc vocabulary additions that coalesce multi-token character sequences for low-resource languages, reducing inference cost without retraining.
This entry links to an external website rather than a GitHub repository, so there's no README to display here.