1.15.0

Dataset Changes

• Update: JNLBA - add tags names by @bhavitvyamalik in https://github.com/huggingface/datasets/pull/3092
• Update: OpenSLR - add SLR83 to OpenSLR by @tyrius02 in https://github.com/huggingface/datasets/pull/3125 and https://github.com/huggingface/datasets/pull/3176
• Update: RONEC - update to v2 by @dumitrescustefan in https://github.com/huggingface/datasets/pull/3184
• Fix: Arabic Billion Words - Fix script to return all data by @albertvillanova in https://github.com/huggingface/datasets/pull/3136
• Fix: HLGD - fix label mapping by @VictorSanh in https://github.com/huggingface/datasets/pull/3180

Dataset Features

• Allow dynamic first dimension for ArrayXD by @rpowalski in https://github.com/huggingface/datasets/pull/2891
• add multi-proc in to_csv by @bhavitvyamalik in https://github.com/huggingface/datasets/pull/2896
• QOL improvements: auto-flatten_indices and desc in map calls by @mariosasko in https://github.com/huggingface/datasets/pull/3196

Dataset Cards

• Fill in dataset card for NCBI disease dataset by @edugp in https://github.com/huggingface/datasets/pull/3115

Metrics Changes

• New: metric for the MATH dataset (competition_math). by @hacobe in https://github.com/huggingface/datasets/pull/3020
• New: Google BLEU (aka GLEU) metric by @slowwavesleep in https://github.com/huggingface/datasets/pull/3108
• New: TER by @BramVanroy in https://github.com/huggingface/datasets/pull/3153
• New: ChrF(++) by @BramVanroy in https://github.com/huggingface/datasets/pull/3187

General improvements and bug fixes

• Correctly update metadata to preserve features when concatenating datasets with axis=1 by @mariosasko in https://github.com/huggingface/datasets/pull/3120
• Fixes to to_tf_dataset by @Rocketknight1 in https://github.com/huggingface/datasets/pull/3085
• Add security policy to the project by @albertvillanova in https://github.com/huggingface/datasets/pull/2958
• Update doc links to point to new docs by @mariosasko in https://github.com/huggingface/datasets/pull/3116
• Fix caching bugs by @mariosasko in https://github.com/huggingface/datasets/pull/3141
• Fix numpy deprecation warning for ragged tensors by @lhoestq in https://github.com/huggingface/datasets/pull/3137
• Fixed: duplicate parameter and missing parameter in docstring by @PanQiWei in https://github.com/huggingface/datasets/pull/3157
• Fix some typos in the documentation by @h4iku in https://github.com/huggingface/datasets/pull/3152
• Fix string encoding for Value type by @lhoestq in https://github.com/huggingface/datasets/pull/3158
• Fix CLI test to ignore verfications when saving infos by @albertvillanova in https://github.com/huggingface/datasets/pull/3147
• Make inspect.get_dataset_config_names always return a non-empty list by @albertvillanova in https://github.com/huggingface/datasets/pull/3159
• Fix issue with filelock filename being too long on encrypted filesystems by @mariosasko in https://github.com/huggingface/datasets/pull/3173
• Asserts replaced by exceptions (huggingface#3171) by @joseporiolayats in https://github.com/huggingface/datasets/pull/3174
• Preserve ordering in zip_dict by @mariosasko in https://github.com/huggingface/datasets/pull/3170
• Don't memoize strings when hashing since two identical strings may have different python ids by @lhoestq in https://github.com/huggingface/datasets/pull/3182
• Re-add faiss to windows testing suite by @BramVanroy in https://github.com/huggingface/datasets/pull/3151
• Add missing docstring to DownloadConfig by @mariosasko in https://github.com/huggingface/datasets/pull/3183
• More efficient nested features encoding by @eladsegal in https://github.com/huggingface/datasets/pull/3124
• Fix optimized encoding for arrays by @lhoestq in https://github.com/huggingface/datasets/pull/3197