ucto

github.com ↗

Unicode-aware regular-expression based tokenizer for various languages. Tool and C++ library. Supports FoLiA format.