v0.13.4.rc1 — Tokenizers

What's Changed

Update all GH Actions with dependency on actions/checkout by @mfuntowicz in https://github.com/huggingface/tokenizers/pull/1256
Parallelize unigram trainer by @mishig25 in https://github.com/huggingface/tokenizers/pull/976
Update unigram/trainer.rs by @chris-ha458 in https://github.com/huggingface/tokenizers/pull/1257
Fixing broken link. by @Narsil in https://github.com/huggingface/tokenizers/pull/1268
fix documentation regarding regex by @chris-ha458 in https://github.com/huggingface/tokenizers/pull/1264
Update Cargo.toml by @chris-ha458 in https://github.com/huggingface/tokenizers/pull/1266
Update README.md - Broken link by @sbhavani in https://github.com/huggingface/tokenizers/pull/1272
[doc build] Use secrets by @mishig25 in https://github.com/huggingface/tokenizers/pull/1273
Improve error for truncation with too high stride by @boyleconnor in https://github.com/huggingface/tokenizers/pull/1275
Add unigram bytefallback by @ArthurZucker in https://github.com/huggingface/tokenizers/pull/1217
revise type specification by @hiroshi-matsuda-rit in https://github.com/huggingface/tokenizers/pull/1289
Bump tough-cookie from 4.0.0 to 4.1.3 in /bindings/node by @dependabot in https://github.com/huggingface/tokenizers/pull/1291
Update path name: master -> main by @bact in https://github.com/huggingface/tokenizers/pull/1292
import Tuple from typing by @kellymarchisio in https://github.com/huggingface/tokenizers/pull/1295
Fixing clippy warnings on 1.71. by @Narsil in https://github.com/huggingface/tokenizers/pull/1296
Bump word-wrap from 1.2.3 to 1.2.4 in /bindings/node by @dependabot in https://github.com/huggingface/tokenizers/pull/1299
feat: Added CITATION.cff. by @SamuelLarkin in https://github.com/huggingface/tokenizers/pull/1302
Single warning for holes. by @Narsil in https://github.com/huggingface/tokenizers/pull/1303
Give error when initializing tokenizer with too high stride by @boyleconnor in https://github.com/huggingface/tokenizers/pull/1306
Handle when precompiled charsmap is empty by @kellymarchisio in https://github.com/huggingface/tokenizers/pull/1308
Derive clone for TrainerWrapper by @jonatanklosko in https://github.com/huggingface/tokenizers/pull/1317
CD backports by @chris-ha458 in https://github.com/huggingface/tokenizers/pull/1318
0.13.4.rc1 by @Narsil in https://github.com/huggingface/tokenizers/pull/1319

New Contributors

@sbhavani made their first contribution in https://github.com/huggingface/tokenizers/pull/1272
@boyleconnor made their first contribution in https://github.com/huggingface/tokenizers/pull/1275
@hiroshi-matsuda-rit made their first contribution in https://github.com/huggingface/tokenizers/pull/1289
@bact made their first contribution in https://github.com/huggingface/tokenizers/pull/1292
@kellymarchisio made their first contribution in https://github.com/huggingface/tokenizers/pull/1295
@SamuelLarkin made their first contribution in https://github.com/huggingface/tokenizers/pull/1302
@jonatanklosko made their first contribution in https://github.com/huggingface/tokenizers/pull/1317

Full Changelog: https://github.com/huggingface/tokenizers/compare/v0.13.4-rc2...v0.13.4.rc1