v0.19.0

What's Changed

• chore: Remove CLI - this was originally intended for local development by @bryantbiggs in https://github.com/huggingface/tokenizers/pull/1442
• [remove black] And use ruff by @ArthurZucker in https://github.com/huggingface/tokenizers/pull/1436
• Bump ip from 2.0.0 to 2.0.1 in /bindings/node by @dependabot in https://github.com/huggingface/tokenizers/pull/1456
• Added ability to inspect a 'Sequence' decoder and the AddedVocabulary. by @eaplatanios in https://github.com/huggingface/tokenizers/pull/1443
• 🚨🚨 BREAKING CHANGE 🚨🚨: (add_prefix_space dropped everything is using prepend_scheme enum instead) Refactor metaspace by @ArthurZucker in https://github.com/huggingface/tokenizers/pull/1476
• Add more support for tiktoken based tokenizers by @ArthurZucker in https://github.com/huggingface/tokenizers/pull/1493
• PyO3 0.21. by @Narsil in https://github.com/huggingface/tokenizers/pull/1494
• Remove 3.13 (potential undefined behavior.) by @Narsil in https://github.com/huggingface/tokenizers/pull/1497
• Bumping all versions 3 times (ty transformers :) ) by @Narsil in https://github.com/huggingface/tokenizers/pull/1498
• Fixing doc. by @Narsil in https://github.com/huggingface/tokenizers/pull/1499

Full Changelog: https://github.com/huggingface/tokenizers/compare/v0.15.2...v0.19.0