releases.shpreview
Hugging Face/Tokenizers/v0.11.2

v0.11.2

Rust v0.11.2

February 28, 2022TokenizersView original ↗
$npx -y @buildinternet/releases show rel_NXPODcY0nPrHv5Ig6XqKH
  • [#919] Fixing single_word AddedToken. (regression from 0.11.2)
  • [#916] Deserializing faster added_tokens by loading them in batch.

Fetched April 7, 2026