Bumping 3 versions because of this: https://github.com/huggingface/transformers/blob/60dea593edd0b94ee15dc3917900b26e3acfbbee/setup.py#L177
remove black] And use ruff by @ArthurZucker in https://github.com/huggingface/tokenizers/pull/1436AddedVocabulary. by @eaplatanios in https://github.com/huggingface/tokenizers/pull/1443Full Changelog: https://github.com/huggingface/tokenizers/compare/v0.15.2...v0.19.0rc0
Fetched April 7, 2026