releases.shpreview
Hugging Face/Tokenizers/python-v0.9.4

python-v0.9.4

Python v0.9.4

November 10, 2020TokenizersView original ↗
$npx -y @buildinternet/releases show rel_-ky4bAQglmqlj6h4dMGhK

Fixed

  • [#492]: Fix from_file on BertWordPieceTokenizer
  • [#498]: Fix the link to download sentencepiece_model_pb2.py
  • [#500]: Fix a typo in the docs quicktour

Changed

  • [#506]: Improve Encoding mappings for pairs of sequence

Fetched April 7, 2026