2.17.0 — Datasets — releases.sh

Dataset Features

[WebDataset] Audio support and bug fixes by @lhoestq in https://github.com/huggingface/datasets/pull/6573
Add concurrent loading of shards to datasets.load_from_disk by @kkoutini in https://github.com/huggingface/datasets/pull/6464
Support data_dir parameter in push_to_hub by @albertvillanova in https://github.com/huggingface/datasets/pull/6634
Support push_to_hub without org/user to default to logged-in user by @albertvillanova in https://github.com/huggingface/datasets/pull/6629
Allow concatenation of datasets with mixed structs by @Dref360 in https://github.com/huggingface/datasets/pull/6587

General improvements and bug fixes

Fix parallel downloads for datasets without scripts by @lhoestq in https://github.com/huggingface/datasets/pull/6551
Fix imagefolder with one image by @lhoestq in https://github.com/huggingface/datasets/pull/6556
Fix tests based on datasets that used to have scripts by @lhoestq in https://github.com/huggingface/datasets/pull/6574
remove eli5 test by @lhoestq in https://github.com/huggingface/datasets/pull/6583
[IterableDataset] Fix drop_last_batchin map after shuffling or sharding by @lhoestq in https://github.com/huggingface/datasets/pull/6575
Support standalone yaml by @lhoestq in https://github.com/huggingface/datasets/pull/6557
Drop redundant None guard. by @xkszltl in https://github.com/huggingface/datasets/pull/6596
fix os.listdir return name is empty string by @d710055071 in https://github.com/huggingface/datasets/pull/6581
Fix CI: pyarrow 15, pandas 2.2 and sqlachemy by @lhoestq in https://github.com/huggingface/datasets/pull/6617
Dedicated RNG object for fingerprinting by @mariosasko in https://github.com/huggingface/datasets/pull/6606
Migrate from setup.cfg to pyproject.toml by @mariosasko in https://github.com/huggingface/datasets/pull/6619
keep more info in DatasetInfo.from_merge #6585 by @JochenSiegWork in https://github.com/huggingface/datasets/pull/6586
Read GeoParquet files using parquet reader by @weiji14 in https://github.com/huggingface/datasets/pull/6508
Use schema metadata only if it matches features by @lhoestq in https://github.com/huggingface/datasets/pull/6616
Raise error on bad split name by @lhoestq in https://github.com/huggingface/datasets/pull/6626
Disable tqdm bars in non-interactive environments by @mariosasko in https://github.com/huggingface/datasets/pull/6627
Add with_rank param to Dataset.filter by @mariosasko in https://github.com/huggingface/datasets/pull/6608
Bump max range of dill to 0.3.8 by @ringohoffman in https://github.com/huggingface/datasets/pull/6630
Fix filelock: use current umask for filelock >= 3.10 by @lhoestq in https://github.com/huggingface/datasets/pull/6631
Faster webdataset streaming by @lhoestq in https://github.com/huggingface/datasets/pull/6578
Multi gpu docs by @lhoestq in https://github.com/huggingface/datasets/pull/6550
dataset viewer requires no-script by @severo in https://github.com/huggingface/datasets/pull/6633
Make split slicing consistent with list slicing by @mariosasko in https://github.com/huggingface/datasets/pull/5891
Do not use Parquet exports if revision is passed by @albertvillanova in https://github.com/huggingface/datasets/pull/6555
Make CLI test support multi-processing by @albertvillanova in https://github.com/huggingface/datasets/pull/6628
Fix reload cache with data dir by @lhoestq in https://github.com/huggingface/datasets/pull/6632
Fix array cast/embed with null values by @mariosasko in https://github.com/huggingface/datasets/pull/6283
Faster column validation and reordering by @psmyth94 in https://github.com/huggingface/datasets/pull/6636
Better multi-gpu example by @lhoestq in https://github.com/huggingface/datasets/pull/6646
Fix missing info when loading some datasets from Parquet export by @lhoestq in https://github.com/huggingface/datasets/pull/6635
Minor multi gpu doc improvement by @lhoestq in https://github.com/huggingface/datasets/pull/6649
Document usage of hfh cli instead of git by @lhoestq in https://github.com/huggingface/datasets/pull/6648

New Contributors

@xkszltl made their first contribution in https://github.com/huggingface/datasets/pull/6596
@kkoutini made their first contribution in https://github.com/huggingface/datasets/pull/6464
@JochenSiegWork made their first contribution in https://github.com/huggingface/datasets/pull/6586
@weiji14 made their first contribution in https://github.com/huggingface/datasets/pull/6508
@ringohoffman made their first contribution in https://github.com/huggingface/datasets/pull/6630
@psmyth94 made their first contribution in https://github.com/huggingface/datasets/pull/6636

Full Changelog: https://github.com/huggingface/datasets/compare/2.16.1...2.17.0