Hugging Face

Datasets Features

Agent traces

• Parse Agent traces messages for SFT using teich by @lhoestq in https://github.com/huggingface/datasets/pull/8232

  • Agent traces from claude_code/pi/codex and others can now be loaded with load_dataset
  • Using the teich library (new optional dependency), traces are parsed to messages to enable training on traces using e.g. trl
  • Load the data:

  >>> from datasets import load_dataset
  >>> ds = load_dataset("lhoestq/agent-traces-example", split="train")
  >>> ds[0]["messages"]
  [{'role': 'user', 'content': 'Download a random dataset from Hugging Face, use DuckDB to inspect it, and come back with a short report about it. Be concise and include: dataset name, what files/format you found, row count or rough size if you can determine it,...'
   ...]

  • Train on agent traces:

  trl sft --dataset-name lhoestq/agent-traces-example ...

  • find all the Agent traces datasets on HF here: https://huggingface.co/datasets?format=format:agent-traces&sort=trending

Next-level shuffling in streaming mode

• Use multiple input shards for shuffle buffer by @lhoestq in https://github.com/huggingface/datasets/pull/8194

  ds = load_dataset(..., streaming=True)
  ds = ds.shuffle(seed=42)
  # or configure local buffer shuffling manually, default is:
  ds = ds.shuffle(seed=42, buffer_size=1000, max_buffer_input_shards=10)

  before👎: <img width="1061" height="529" alt="image" src="https://github.com/user-attachments/assets/dd85a11b-72fc-474c-8faa-33eda779a4fb" />

  after✨: <img width="1057" height="523" alt="image" src="https://github.com/user-attachments/assets/77acc8d3-61d4-4e57-bc87-43fcd3c2e8f3" />

  toy example comparison

  from datasets import IterableDataset
  
  ds = IterableDataset.from_dict({"i": range(123_456_789)}, num_shards=1024)
  ds = ds.shuffle(seed=42)
  
  print("Cold start ids:")
  print(list(ds.take(10)["i"]))
  print("Nominal regime ids:")
  print(list(ds.skip(10_000).take(10)["i"]))

  before👎:

  Cold start ids:
  [6148853, 6149537, 6149418, 6149202, 6149197, 6149622, 6148849, 6149461, 6148965, 6148858]
  Nominal regime ids:
  [6149537, 6149418, 6149202, 6149197, 6149622, 6148849, 6149461, 6148965, 6148858, 6149290]

  after✨:

  Cold start ids:
  [7836668, 9283505, 95847927, 482299, 9283471, 482341, 112003312, 59920157, 43764666, 95847871]
  Nominal regime ids:
  [9283505, 95847927, 482299, 9283471, 482341, 112003312, 59920157, 43764666, 95847871, 16758448]

  Note: ds.state_dict() and ds.load_state_dict() are still supported for this improved shuffling :) enabling dataset checkpointing

  Note 2: it uses threads to fetch the first examples in parallel from the input shards

  Note 3: This is a BREAKING CHANGE: the default shuffling mechanism now uses multiple input shards. You can get the old mechanism by passing max_buffer_input_shards=1 to IterableDataset.shuffle()

New batching features for robotics datasets

• Add batch(by_column=...) by @lhoestq in https://github.com/huggingface/datasets/pull/8172

  from datasets import Dataset
  
  ds = Dataset.from_dict({"episode": [0] * 10 + [1] * 10, "frame": list(range(10)) * 2})
  # ds = ds.to_iterable_dataset()
  ds = ds.batch(by_column="episode")
  for x in ds:
      print(x)
  # {'episode': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'frame': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]}
  # {'episode': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'frame': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]}

New supported formats

• Add Apache Iceberg format support by @frankliee in https://github.com/huggingface/datasets/pull/8148
• feat: add TsFile (Apache IoTDB) packaged builder with per-device wide format by @JackieTien97 in https://github.com/huggingface/datasets/pull/8160
• feat: add 3D mesh support and MeshFolder builder by @Vinay-Umrethe in https://github.com/huggingface/datasets/pull/8055
• Add .conll / .conllu dataset format loader (CoNLL-2003 / 2000 / U) by @CrypticCortex in https://github.com/huggingface/datasets/pull/8219

Other improvements and bug fixes

• Pass library_name/version to HfApi in dataset push and delete paths by @davanstrien in https://github.com/huggingface/datasets/pull/8161
• Fix storage_options lookup for streaming Lance datasets by @ericjaebeom in https://github.com/huggingface/datasets/pull/8166
• add agent trace prompt, sent_at, count fields by @cfahlgren1 in https://github.com/huggingface/datasets/pull/8163
• fix: add num_proc argument to Dataset.to_sql by @EricSaikali in https://github.com/huggingface/datasets/pull/7791
• Support fsspec 2026.4.0 by @lhoestq in https://github.com/huggingface/datasets/pull/8175
• Fix Parquet streaming hangs at the end of script by @lhoestq in https://github.com/huggingface/datasets/pull/8176
• ClassLabel docs: Correct value for unknown labels by @l-uuz in https://github.com/huggingface/datasets/pull/7645
• fix parquet reshard by @lhoestq in https://github.com/huggingface/datasets/pull/8193
• Fix parquet columns arg by @lhoestq in https://github.com/huggingface/datasets/pull/8210
• update readme by @lhoestq in https://github.com/huggingface/datasets/pull/8208
• update single seg repos in ci by @lhoestq in https://github.com/huggingface/datasets/pull/8213
• Fix single lance file form pylance 7.0 by @lhoestq in https://github.com/huggingface/datasets/pull/8225
• fix(map): fix progress bar exceeding total when load_from_cache_file=False by @Nitin-Rajasekar in https://github.com/huggingface/datasets/pull/8170
• fix: embed_external_files=True for mesh support by @Vinay-Umrethe in https://github.com/huggingface/datasets/pull/8224
• Fix iterable skip over full Arrow blocks by @my17th2 in https://github.com/huggingface/datasets/pull/8236
• Keep None as a real null in Json() columns instead of the string "null" by @adityasingh2400 in https://github.com/huggingface/datasets/pull/8231
• Support composed splits in streaming datasets by @lanarkite99 in https://github.com/huggingface/datasets/pull/8220

New Contributors

• @ericjaebeom made their first contribution in https://github.com/huggingface/datasets/pull/8166
• @EricSaikali made their first contribution in https://github.com/huggingface/datasets/pull/7791
• @l-uuz made their first contribution in https://github.com/huggingface/datasets/pull/7645
• @CrypticCortex made their first contribution in https://github.com/huggingface/datasets/pull/8219
• @frankliee made their first contribution in https://github.com/huggingface/datasets/pull/8148
• @Vinay-Umrethe made their first contribution in https://github.com/huggingface/datasets/pull/8055
• @Nitin-Rajasekar made their first contribution in https://github.com/huggingface/datasets/pull/8170
• @JackieTien97 made their first contribution in https://github.com/huggingface/datasets/pull/8160
• @my17th2 made their first contribution in https://github.com/huggingface/datasets/pull/8236
• @adityasingh2400 made their first contribution in https://github.com/huggingface/datasets/pull/8231
• @lanarkite99 made their first contribution in https://github.com/huggingface/datasets/pull/8220

Full Changelog: https://github.com/huggingface/datasets/compare/4.8.5...5.0.0

🖥️ Unified hf cp command

A single hf cp command now handles all file-copy workflows (upload a local file, download from the Hub, or copy between two remote locations) with consistent hf:// URI syntax for both repositories and buckets. It is also available as hf repos cp and hf buckets cp; all three aliases are identical, so you can use whichever reads best for your workflow. You can stream from stdin (-) or to stdout (-), and a trailing / on the source path gives you rsync-style semantics (copy the folder contents, not the folder itself). Note that remote-to-remote copies only work within the same storage region, and bucket-to-repo is not yet supported.

# Upload a local file to a repo
hf cp ./model.safetensors hf://username/my-model/model.safetensors

# Download a file to stdout
hf cp hf://username/my-model/config.json - | jq .

# Copy between two Hub repos
hf cp hf://username/source-model/config.json hf://username/dest-model/config.json

📚 Documentation: CLI guide — Copy files

• [CLI] Add unified hf cp command (aliased as hf repos cp and hf buckets cp) by @Wauplin in #4295

🥚 Easter egg:explore your storage usage

• [CLI] Easter egg: city skyline in hf repos ls by @Wauplin in #4287

🔗 Paste web URLs directly

parse_hf_uri now accepts Hugging Face web URLs so you can paste a link straight into the CLI or the library and it "just works".

# Copy-paste a URL from the website
hf cp https://huggingface.co/nvidia/LocateAnything-3B/blob/main/config.json - | jq '.architectures'

📚 Documentation: HF URIs — Web URLs

• [URIs] Parse web URLs in parse_hf_uri + add HfUri.to_url by @Wauplin in #4296

🚨 Breaking change

On Lustre, GPFS, and some NFS mounts, flock(2) silently succeeds for every caller, which means filelock provides no mutual exclusion. When multiple hf_hub_download calls race for the same file, they can append to the same .incomplete file and silently corrupt the blob cache. This release fixes that by always downloading to a fresh temporary file instead of resuming an incomplete one, making the download path safe even when file locking is broken. filelock is still used as a "best-effort" hint to avoid unnecessary duplicate downloads, but correctness no longer depends on it. This is a breaking change: resuming a previously failed partial download is no longer possible. However, file resumability was already a niche use case only applicable when hf_xet is disabled.

• [Fix] Make concurrent downloads safe even when file locking is broken by @Wauplin in #4306

🖥️ CLI

• [CLI] inline enum choices in the generated CLI skill by @hanouticelina in #4299

🐛 Bug and typo fixes

• Fix ~ user home not expanded in local_dir and cache_dir on file download by @Wauplin in #4293
• Do not fail on repo/bucket creation if HTTP 401 and already exists by @Wauplin in #4294
• Fix umask probe writing tmp file outside download dir by @Wauplin in #4305

📖 Documentation

• [Docs] Document missing endpoint and template_str parameters by @aicayzer in #4298
• [Docs] Document missing parameters in hf_hub_url and preupload_lfs_files by @aicayzer in #4300
• [Docs] Mention storage region limitation for server-side copy by @Wauplin in #4302

🏗️ Internal

• Post-release: bump version to 1.18.0.dev0 by @huggingface-hub-bot[bot] in #4291
• Bump the actions group with 2 updates by @dependabot[bot] in #4309

There was a big bug in the model conversion of models related to clip, this affected models like sam3 and others. Please make sure to update 🙏

• Fix conversion for clip models by @zucchini-nlp (#46406)

Full Changelog: https://github.com/huggingface/transformers/compare/v5.10.1...v5.10.2

v5.10.0 was yanked as we publish on a corrupted branch. Sorry everyone, this happens when we rush a release!!!

New Model additions

Gemma4 unified+ Gemma4 MTP

Gemma 4 12B Unified is an encoder-free multimodal model with pretrained and instruction-tuned variants. Unlike standard Gemma 4, which uses dedicated encoder towers, Gemma 4 12B Unified projects raw inputs directly into the language model's embedding space through lightweight linear pipelines. This results in a simpler architecture while maintaining strong multimodal performance.

Key differences from standard Gemma 4:

• No Vision Tower: Raw pixel patches are projected directly into LM space via a Dense + LayerNorm pipeline with factorized 2D positional embeddings, replacing the vision encoder.
• No Audio Tower: Raw 16 kHz waveform samples are chunked into fixed-length frames and projected through a simple RMSNorm → Linear pipeline, replacing the mel spectrogram + Conformer encoder.
• Shared Multimodal Pipeline: Both vision and audio use the same Gemma4UnifiedMultimodalEmbedder (RMSNorm → Linear) for the final projection to text hidden space.

You can find the original Gemma 4 12B Unified checkpoints under the Gemma 4 release.

• who needs encoders? (#46385) by @douglas-reid @sgerrard @vasqu @molbap

Sapiens2

Sapiens2 is a family of high-resolution vision transformers pretrained on ~1 billion curated human images, designed for human-centric computer vision tasks including pose estimation, body-part segmentation, surface normal estimation, and pointmap estimation. The models scale from 0.4B to 5B parameters and train at native 1K resolution, with hierarchical 4K variants for extended spatial reasoning. Sapiens2 achieves substantial improvements over its predecessor with +4 mAP in pose estimation, +24.3 mIoU in body-part segmentation, and 45.6% error reduction in normal estimation.

Links: Documentation | Paper

• Add Sapiens2 Model (#45919) by @guarin in #45919

DeepSeek-OCR-2

DeepSeek-OCR-2 is an OCR-specialized vision-language model built on a distinctive architecture that combines a SAM ViT-B vision encoder with a Qwen2 hybrid attention encoder, connected through an MLP projector to a DeepSeek-V2 Mixture-of-Experts (MoE) language model. The model features a hybrid attention mechanism that applies bidirectional attention over image tokens and causal attention over query tokens, enabling efficient and accurate document understanding. It supports both plain OCR tasks and grounding capabilities with coordinate-aware output for document conversion to markdown format.

Links: Documentation

• Add Deepseek-OCR-2 model (#45075) by @thisisiron in #45075

Mellum

Mellum is a code-focused Mixture-of-Experts language model developed by JetBrains. It is derived from the Qwen3-MoE architecture with per-layer-type RoPE and interleaved sliding window attention. The model has 12B total parameters with 2.5B active parameters per token, using 64 routed experts with 8 activated per token across 28 layers.

Links: Documentation

• feat: Add support for JetBrains' Mellum v2 code generation model (#46112) by @shadeMe in #46112

Breaking changes

The Gemma4 vision pooler now casts inputs to float32 before scaling to prevent float16 overflow (inf saturation) with large checkpoints, which may cause minor numerical differences in outputs for users running Gemma-4 vision models in float16.

• 🚨 Fix float16 overflow in Gemma4 vision pooler (#46277) by @Bluear7878

Audio Language Models (ALMs) now have a dedicated base model class without a language modeling head, aligning them with the design of Vision Language Models (VLMs); users relying on the previous model class structure should update their code to use the new base model class where appropriate.

• 🚨 [ALM] Add base model without head (#45534) by @eustlb

Parallelization

This release includes numerous bug fixes for model parallelism across multiple models (Gemma4, AltCLIP, ChineseClip, Blip-2, Whisper, Ovis2, Moshi) and parallel execution strategies, including fixes for tensor parallelism (TP), expert parallelism (EP), beam search under model parallel settings, and loss over-counting under TP/EP configurations. The continuous batching manager was also reworked for clearer control flow and improved TP race condition handling, and FSDP initialization via from_pretrained was introduced.

• Fix dsv4 dequant + tp/ep (#46378) by @IlyasMoutawwakil in [#46378]
• [CB] [Major] Rework manager to have clearer control flow + handle TP (#46070) by @remi-or in [#46070]
• fix series of bugs for model parallel beam search (#46280) by @kaixuanliu in [#46280]
• Fix model parallel issue for altclip model and ChineseClip model (#45487) by @kaixuanliu in [#45487]
• Model parallel fix (#46230) by @kaixuanliu in [#46230]
• [Revert] FSDP+Dtensor refactor related changes (#46246) by @vasqu in [#46246]
• Fix model parallel bugs for Gemma4 (#45817) by @kaixuanliu in [#45817]
• init FSDP through from_pretrained (#46102) by @3outeille in [#46102]
• fix model parallel device mismatch issue in create_bidirectional_mask (#46221) by @kaixuanliu in [#46221]
• Trainer.compute_loss: fix loss over-counting under TP and EP-as-TP (#45994) by @AmineDiro in [#45994]
• Fix caching allocator warmup byte estimation for EP model loading (#46149) by @sywangyi in [#46149]

Cache

Fixed a regression in encoder-decoder cache initialization where the decoder config was incorrectly applied to the cross-attention cache, and resolved a RuntimeError caused by buffer size limits when warming up the cache on MPS devices. Additional test infrastructure improvements were made to support read-only cache environments used in CI.

• fix: cache warmup RuntimeError on mps (#46239) by @McPatate in [#46239]
• Make more tests work with read-only cache (#46299) by @ydshieh in [#46299]
• Update a test to avoid writing to the default xet cache (#46250) by @ydshieh in [#46250]
• Fix a regression in encoder-decoder generation cache initialization (#46111) by @kaixuanliu in [#46111]

Quantization

Added support for DeepGEMM BF16, mixed FP8/FP4, and MegaMoE quantization via a grouped linear refactor, while fixing two bugs: an FP8 MoE reverse substring issue affecting DSv4 initialization, and a BitsAndBytes 4-bit/8-bit quantization bug that silently dropped chunked tensors from one-to-many weight converters.

• DeepGEMM BF16 + mixed FP8/FP4 + MegaMoE + refactor (#45634) by @IlyasMoutawwakil in [#45634]
• Fix fp8 moe reverse substring (#46265) by @ArthurZucker in [#46265]
• Fix bnb 4bit/8bit quantization drop chunked tensors bug (#46210) by @kaixuanliu in [#46210]

Bugfixes and improvements

• Fix wrong changes produced by style/repo. check bot (#46371) by @ydshieh in [#46371]
• Fix path traversal when saving Bark voice preset embeddings (#46237) by @LinZiyuu in [#46237]
• Pass library_name/version to Hub calls via a shared HfApi (#46318) by @Wauplin in [#46318]
• docs: update ACL Anthology URL in CITATION.cff (#46352) by @irfaan101 in [#46352]
• [docs] contributing (#45465) by @stevhliu in [#45465]
• [docs] Romanian translation of contributing.md, modular_transformers.md, multimodal_processing.md, add_vision_processing_components.md, add_audio_processing_components.md, modeling_rules.md, model_output_tracing.md, auto_docstring.md, testing.md, pr_checks.md and add_new_model.md . (#46345) by @filipinescu in [#46345]
• [docs] xpu continuous batching (#46334) by @stevhliu in [#46334]
• Fix incorrect attribute mapping relationships in GLM MoE DSA Config (#46338) by @Dovis01 in [#46338]
• Fix grammar typos in Whisper documentation (#46336) by @calliec-1223 in [#46336]
• [docs] update num_items_in_batch for causal LMs (#46335) by @stevhliu in [#46335]
• Update compressed tensors minimum version (#46342) by @SunMarc in [#46342]
• Fix _is_package_available reporting available without a version (#46125) by @blipbyte in [#46125]
• remove sec (#46346) by @ydshieh in [#46346]
• fix: include transitive relative imports when loading from local directory (#46022) by @trducng in [#46022]
• perf(feature_extraction_sequence): skip re-splitting already-batched numpy arrays in pad() (#46329) by @Anai-Guo in [#46329]
• [Zamba] Support attn_implementation dispatch (#46317) by @YangKai0616 in [#46317]
• Fix TestAppRoutes test failures caused by deprecated asyncio.get_event_loop() on Python 3.10+ (#46340) by @ydshieh in [#46340]
• [Qwen3VL] Fix video token placeholder: use self.video_token instead of hardcoded "<|placeholder|>" (#46296) by @kpal002 in [#46296]
• chore(linter): fixes for rule 16 (#46023) by @tarekziade in [#46023]
• [docs] Romanian translation of weightconverter.md, models.md, custom_models.md, monkey_patching.md, fusion_mapping.md, how_to_hack_models.md, model_sharing.md and serialization.md. (#46309) by @filipinescu in [#46309]
• Normalize CUDA OOM errors when comparing commit failures in check_bad_commit (#46322) by @ydshieh in [#46322]
• Fix unhandled exception noise from background safetensors conversion thread (#45752) by @dhruv7477 in [#45752]
• Add Expectations for pipeline token classification tests (#46151) by @kaixuanliu in [#46151]
• [docs] fix auto-add release dates (#46283) by @zucchini-nlp in [#46283]
• Separate pip command syntax for notebook and CLI tabs in Quickstart (#46243) by @pvelayudhan in [#46243]
• Romanian translation of README.md, index.md, installation.md, _config.py and quicktour.md. (#46166) by @filipinescu in [#46166]
• Fall back to flat kwarg when modality dict is passed without it (#46195) by @Ace3Z in [#46195]
• Fix load_adapter OOM caused by full-model warmup sizing (#46145) by @Yooniel in [#46145]
• Replace assert with raise ImportError for optuna/ray dependency checks (#46263) by @SebTardif in [#46263]
• chore(linter): respect TRF017 modeling rule (#46260) by @tarekziade in [#46260]
• Delete dead code in qwen-vl series (#45827) by @zucchini-nlp in [#45827]
• qa: fix ty caching and align CI with local run (#46278) by @tarekziade in [#46278]
• Guard DeviceMesh import in continuous batching (#46205) by @danyalahmed1995 in [#46205]
• Processor compatibility with vLLM (#46258) by @zucchini-nlp in [#46258]
• Fix PR CI workflow cancellation condition (#46276) by @ydshieh in [#46276]
• [fix] toctree (#46106) by @stevhliu in [#46106]
• add more generic support for distributed trainer tests (#46109) by @kaixuanliu in [#46109]
• add XPU Expectations for florence2 and lfm2_vl model test (#46275) by @kaixuanliu in [#46275]
• Fix StaticCache building an empty layer list when num_kv_shared_layers == 0 (#46235) by @tengomucho in [#46235]
• Fix inverted assertion in remove_handler (#46227) by @SebTardif in [#46227]
• [ShieldGemma2] Support attn_implementation dispatch (#46069) by @YangKai0616 in [#46069]
• [Gemma4] Replace one-hot matmul with F.embedding in position embeddings (#46176) by @Sriniketh24 in [#46176]
• fix: kosmos2.5: properly expand embeddings table (#45835) by @nunq in [#45835]
• find pytest launch error in torch 2.13.0.dev20260526 (#46252) by @sywangyi in [#46252]
• [Test][Kosmos2.5] Add XPU expectations for integration tests (#46135) by @YangKai0616 in [#46135]
• Support FA2 flash_attn_with_kvcache for XPU continuous batching (#46028) by @YangKai0616 in [#46028]
• [Configs] Fix layer type validation to include its mlp counterpart (#46220) by @vasqu in [#46220]
• Fix num_items_in_batch over-counting for causal LM losses (#46204) by @qgallouedec in [#46204]
• RF-DETR doc fixes (#46244) by @merveenoyan in [#46244]
• Use main instead of commit SHA for now (#46241) by @ydshieh in [#46241]
• Enable push event (to main) for PR CI workflow (#46240) by @ydshieh in [#46240]
• fix(hrm_text): Add XPU Expectations for tests (#46214) by @kaixuanliu in [#46214]
• [deepseek_v4] keep hc_head / sinks / position_bias in fp32 (#46198) by @ArthurZucker in [#46198]
• Fix FSDP2 and distributed checkpointing imports for older PyTorch versions (#46141) by @ryota-komatsu in [#46141]
• Fix Gemma4 Array Mask Indexing (#46203) by @petecao in [#46203]
• utils: handle flash_attn missing from importlib packages_distributions without crashing (#45524) by @SAY-5 in [#45524]
• [AMD CI] revert AMD mi325 hf-workflows ref from SHA back to @main (#46213) by @Abdennacer-Badaoui in [#46213]
• [GLM-4.6V] Update with GLM-GA Processor (#46184) by @zRzRzRzRzRzRzR in [#46184]
• update xpu expectation for falcon mamba (#46086) by @sywangyi in [#46086]
• chore: enable Dependabot weekly GitHub Actions bumps (#46157) by @hf-dependantbot-rollout[bot] in [#46157]
• Fix Gemma4 use_bidirectional_attention="all" mask behavior (#46079) by @oliverholworthy in [#46079]
• Fix loading with only 1 device or distributed config (#46197) by @Cyrilvallez in [#46197]
• Fix TypeError on list-typed ignore_keys_at_rope_validation in RoPE config (#46142) by @Charly21r in [#46142]
• Support XPU autocast dtype fallback for FlashAttention (#46199) by @YangKai0616 in [#46199]
• Fix path traversal when saving named chat templates (#46191) by @LinZiyuu in [#46191]
• Fix is_last off-by-one in MaskGenerationPipeline for partial batches (#46136) by @J3r3myPerera in [#46136]
• Fix wrong variable in check_model_type isinstance check (#46080) by @SebTardif in [#46080]
• Enable passing kwargs through RoFormer models (#46171) by @ir2718 in [#46171]
• Update cohere2_moe tp_plan (#46189) by @Cyrilvallez in [#46189]
• Update release tool (#46193) by @Cyrilvallez in [#46193]
• [loading] Fix base_model_prefix issues in conversions (#46067) by @Cyrilvallez in [#46067]
• Bump dev version (#46188) by @Cyrilvallez in [#46188]
• Update self-comment-ci (#46137) by @guarin in [#46137]

Significant community contributions

The following contributors have made significant changes to the library over the last release:

• @filipinescu
  • [docs] Romanian translation of contributing.md, modular_transformers.md, multimodal_processing.md, add_vision_processing_components.md, add_audio_processing_components.md, modeling_rules.md, model_output_tracing.md, auto_docstring.md, testing.md, pr_checks.md and add_new_model.md . (#46345)
  • [docs] Romanian translation of weightconverter.md, models.md, custom_models.md, monkey_patching.md, fusion_mapping.md, how_to_hack_models.md, model_sharing.md and serialization.md. (#46309)
  • Romanian translation of README.md, index.md, installation.md, _config.py and quicktour.md. (#46166)
• @remi-or
  • [CB] [Major] Rework manager to have clearer control flow + handle TP (#46070)
• @thisisiron
  • Add Deepseek-OCR-2 model (#45075)
• @kaixuanliu
  • Add Expectations for pipeline token classification tests (#46151)
  • fix series of bugs for model parallel beam search (#46280)
  • add more generic support for distributed trainer tests (#46109)
  • add XPU Expectations for florence2 and lfm2_vl model test (#46275)
  • Fix model parallel issue for altclip model and ChineseClip model (#45487)
  • Model parallel fix (#46230)
  • fix(hrm_text): Add XPU Expectations for tests (#46214)
  • Fix model parallel bugs for Gemma4 (#45817)
  • Fix bnb 4bit/8bit quantization drop chunked tensors bug (#46210)
  • fix model parallel device mismatch issue in create_bidirectional_mask (#46221)
  • Fix a regression in encoder-decoder generation cache initialization (#46111)
• @shadeMe
  • feat: Add support for JetBrains' Mellum v2 code generation model (#46112)
• @vasqu
  • [Revert] FSDP+Dtensor refactor related changes (#46246)
  • [Configs] Fix layer type validation to include its mlp counterpart (#46220)
• @zRzRzRzRzRzRzR
  • [GLM-4.6V] Update with GLM-GA Processor (#46184)
• @eustlb
  • 🚨 [ALM] Add base model without head (#45534)

What's Changed

• Bump dev version: v1.26.0.dev0 by @albertvillanova in https://github.com/huggingface/smolagents/pull/2273
• Add new possible error message to test of LiteLLM providers w/o key by @albertvillanova in https://github.com/huggingface/smolagents/pull/2275
• Fix typo cleanups across docs, examples, and tests by @luojiyin1987 in https://github.com/huggingface/smolagents/pull/2208
• fix: correct f-string escaping in ToolSerializer.dumps error message by @Ricardo-M-L in https://github.com/huggingface/smolagents/pull/2134
• Revert fix: correct f-string escaping by @albertvillanova in https://github.com/huggingface/smolagents/pull/2291
• Update docstring of LocalPythonExecutor by @albertvillanova in https://github.com/huggingface/smolagents/pull/2292
• Add Exa as a search engine option in WebSearchTool by @10ishq in https://github.com/huggingface/smolagents/pull/2139
• Remove remote WasmExecutor by @albertvillanova in https://github.com/huggingface/smolagents/pull/2321

New Contributors

• @luojiyin1987 made their first contribution in https://github.com/huggingface/smolagents/pull/2208
• @Ricardo-M-L made their first contribution in https://github.com/huggingface/smolagents/pull/2134
• @10ishq made their first contribution in https://github.com/huggingface/smolagents/pull/2139

Full Changelog: https://github.com/huggingface/smolagents/compare/v1.25.0...v1.26.0

📋 Copy files between repositories

You can now copy files or entire folders between different repositories on the Hub — model to model, model to dataset, any combination — without downloading or re-uploading data. CommitOperationCopy accepts src_repo_id and src_repo_type for cross-repo sources, and LFS blobs are deduplicated server-side via the /lfs-files/duplicate endpoint. Non-LFS files are fetched from the source repo and committed as regular payloads. copy_files and hf buckets cp now support repo-to-repo in addition to the existing bucket destinations.

>>> from huggingface_hub import copy_files

# Copy an entire folder
>>> copy_files(
...     "hf://datasets/username/source-dataset/data/",
...     "hf://datasets/username/target-dataset/data/",
... )

• [Copy] Support cross-repo file copies by @Wauplin in #4203

📚 Documentation: Upload guide — Copy files between repositories

🖥️ SSH into a Space with hf spaces ssh

A new hf spaces ssh command opens an SSH session directly into a Space's Dev Mode container. If Dev Mode is not enabled yet, the CLI prompts you to enable it. You can also use --dry-run to print the SSH command without running it, or -i to forward a specific key. Your SSH public key must be registered in your HF user settings.

# SSH into a Space
$ hf spaces ssh username/my-space

# Print the SSH command without running it
$ hf spaces ssh username/my-space --dry-run

• [CLI] Add hf spaces ssh by @gary149 in #4241

📚 Documentation: CLI guide — SSH into a Space | Spaces guide — SSH into a Space

📂 List all your repos with hf repos ls

A new hf repos ls command lists all your repositories — models, datasets, spaces, and buckets — with storage size and percentage of namespace total, sorted by storage usage. It supports --type, --search, --namespace, and --limit (default 30, --limit 0 for all), plus the standard --format family.

# List all your repos
$ hf repos ls

# List all datasets under org with JSON output
$ hf repos ls --namespace my-org --type dataset --limit 0 --format json | jq '.[].id'

• [CLI] Add hf repos ls command by @Wauplin in #4283

📚 Documentation: CLI guide — List repos | Repository guide — List your repositories

📊 CLI tables auto-fit terminal width and right-align numbers

Human-mode CLI tables now use a column-aware algorithm that computes per-column width caps from the actual terminal width, shrinking only the widest columns when needed. Non-TTY output keeps the legacy fixed cap, and --no-truncate bypasses truncation entirely. Numeric columns (all int/float values) are automatically right-aligned.

# Tables adapt to your terminal width
$ hf models ls --search qwen3

# Force full values regardless of width
$ hf models ls --no-truncate

• [CLI] Auto-fit human tables to terminal width by @hanouticelina in #4251
• [CLI] Auto right-align numeric columns in human table output by @Wauplin in #4288

📚 Documentation: CLI guide — Output formatting

🔧 Jobs hardware decoupled from Spaces, auto-synced with Hub API

Jobs now have their own JobHardware enum, independent of SpaceHardware, so the two catalogs can diverge as needed. The old JobHardware dataclass (return type of list_jobs_hardware()) has been renamed to JobHardwareInfo. CLI --flavor / --hardware flags use a new SoftChoice type that shows known values for autocomplete but accepts any string — older CLI versions won't reject new server-side flavors. A daily CI workflow (update-hardware-flavors.yaml) runs utils/check_hardware_flavors.py to sync both enums from the live Hub API and opens a bot PR when something changes.

# Unknown flavors pass through instead of raising a validation error
$ hf jobs run --flavor future-gpu-x99 python:3.12 echo "works with unknown flavors"

• [Jobs] Decouple Job hardware from Spaces, auto-sync enums with Hub API by @Wauplin in #4266

📚 Documentation: Jobs guide

💔 Breaking Change

• [Inference] Remove Together ASR task to drop urllib3 dependency by @Wauplin in #4248
• [CLI] migrate hf jobs to out singleton by @hanouticelina in #4254

Note: The second item changes hf jobs ps and hf jobs scheduled ps to use --format auto|human|agent|json|quiet (removing --format table and -q). JSON output for hf jobs ps now flattens command to a string and status to a stage string.

🖥️ CLI

• [CLI] Migrate extensions, lfs-enable-largefiles, version to out singleton by @hanouticelina in #4284
• [CLI] Fix parent aliases leaking into subcommand headers in CLI reference by @Wauplin in #4253

📊 Jobs

• [Jobs] Add update_job_labels and update_scheduled_job_labels by @Wauplin in #4252
• feat: add rtx-pro-6000 hardware flavors by @christophe-rannou in #4259

📚 Documentation: Jobs guide — Update labels | Jobs guide — Hardware

🔧 Other QoL Improvements

• Fix failing on read-only file system even if the non-existence of file is already cached by @ydshieh in #4274
• [CLI] Add click as an explicit dependency (typer 0.26.0 fix) by @hanouticelina in #4270

📖 Documentation

• [Docs] Document missing parameters in lfs, hf_file_system, and repocard_data by @kratos0718 in #4289

🐛 Bug and typo fixes

• fix local_files_only grammar by @hunterhogan in #4255
• Fix typos in comments and debug log message by @Sanjays2402 in #4278

🏗️ Internal

• Post-release: bump version to 1.17.0.dev0 by @huggingface-hub-bot[bot] in #4247
• ci(release): migrate PyPI publish to Trusted Publishing + attestations by @XciD in #4249
• [CI] Add import check with base dependencies only by @Wauplin in #4250
• [Internal] Update style bot permissions by @hanouticelina in #4260
• chore: enable Dependabot weekly GitHub Actions bumps by @hf-dependantbot-rollout[bot] in #4262
• Bump actions in the actions group by @dependabot[bot] in #4263
• chore: update release.yml by @hf-security-analysis[bot] in #4264
• fix(release.yml): restore env: block on 'Detect version' step by @paulinebm in #4267
• [Internal] Cap typer below 0.26.0 (CLI incompatibility) by @hanouticelina in #4272
• [Internal] Run pinact (lock versions in workflow files) by @Wauplin in #4275
• [Internal] Use Comment bot in model card gh action by @Wauplin in #4277
• [CI] fix collection test by making its name unique by @Wauplin in #4279
• Bump astral-sh/setup-uv from 7.6.0 to 8.1.0 in the actions group by @dependabot[bot] in #4281
• [CLI] Drop legacy printing helpers from _cli_utils.py by @hanouticelina in #4285

A new Base only toggle on the Models page hides every finetune, adapter, merge, and quantization, leaving just the original base models. The Other → Model Tree filter also provides the opposite, listing only one relation type across the Hub, like every adapter or every quantized model.

What's Changed

• 🔒 Gate trainer telemetry on an explicit class-name allowlist by @qgallouedec in https://github.com/huggingface/trl/pull/5851

Full Changelog: https://github.com/huggingface/trl/compare/v1.5.0...v1.5.1

• [CI] Cap typer below 0.26.0 (CLI incompatibility) #4272

• [CI] Add click as explicit dependency #4270

Full Changelog: https://github.com/huggingface/huggingface_hub/compare/v1.16.1...v1.16.4

Features

Even more training chat templates

Three more model families gain training-compatible templates with {% generation %} markers (so assistant_only_loss=True just works):

• Phi-3.5 by @DagaBhai in https://github.com/huggingface/trl/pull/5746
• Qwen3-VL by @aazizyan in https://github.com/huggingface/trl/pull/5764
• Qwen3.5 Think / NoThink by @aazizyan in https://github.com/huggingface/trl/pull/5824

Final logits softcapping for async GRPO

The chunked LM-head path used by AsyncGRPOTrainer now supports models that use final_logit_softcapping (notably Gemma 2). _ChunkedLogProbFunction applies logit_scale, optional tanh-based softcapping, and temperature consistently in both forward and backward — softcapped models are no longer rejected.

by @mlarnouhet in https://github.com/huggingface/trl/pull/5691

KTO ↔ DPO alignment continues

Two more cycles closer to KTO graduation:

• Align compute_loss flow by @albertvillanova in https://github.com/huggingface/trl/pull/5810
• Align _compute_loss_liger flow by @albertvillanova in https://github.com/huggingface/trl/pull/5816

Trainer telemetry (opt-out)

_BaseTrainer.__init__ now emits a single anonymous huggingface_hub.send_telemetry ping per trainer instantiation, so we can finally see which trainers / model families / distributed backends are actually being used in practice and prioritize accordingly.

The payload is intentionally minimal — TRL version, trainer class name, model architecture, PEFT yes/no, distributed backend (deepspeed/fsdp/ddp/none), bucketed world size, device type, GPU model when available. No user data, no dataset names, no model paths, no hyperparameter values, never sent in CI / offline / HF_HUB_DISABLE_TELEMETRY mode.

See usage_stats.md for what's collected and how to opt out.

by @qgallouedec in https://github.com/huggingface/trl/pull/5758

Other

• OpenRewardSpec: fix omitting task-scoped tools during rollout binding (fixes #5727) by @rycerzes in https://github.com/huggingface/trl/pull/5729
• Add OpenReward example to the list of examples by @sergiopaniego in https://github.com/huggingface/trl/pull/5752
• Add DDP-2 members to invariant test suite by @qgallouedec in https://github.com/huggingface/trl/pull/5736
• Align and simplify the stable training scripts by @qgallouedec in https://github.com/huggingface/trl/pull/5812
• Replace uv installation script with setup action by @qgallouedec in https://github.com/huggingface/trl/pull/5735

Fixes

• Fix exponential backtracking in qwen3 / qwen3_5 / glm4moe response parsing — GRPOTrainer was hanging indefinitely on truncated <tool_call> blocks (a degenerate case that happens naturally when generation hits max_completion_length mid-tool-call). Rewrote the regex to be non-backtracking — worst case goes from O(2ⁿ) to O(n). By @xodn348 in https://github.com/huggingface/trl/pull/5798
• CUDA memory leak: release BNB dequantization buffers & stale state in OffloadActivations — follow-up to v1.4's activation-offloading leak fix. By @butterwecksolutions in https://github.com/huggingface/trl/pull/5730
• Invalidate ZeRO-3 param coordinator trace in add_hooks by @roycho96 in https://github.com/huggingface/trl/pull/4693
• Fix nested vocab_size for DistillationTrainer and GOLDTrainer by @Beichen-Ma in https://github.com/huggingface/trl/pull/5592
• Fix MPS support in experimental empty_cache() by @jamie-peterson-ml in https://github.com/huggingface/trl/pull/5799
• Fix metric_for_best_model for trainer-specific eval metrics by @qgallouedec in https://github.com/huggingface/trl/pull/5811
• Fix generate_batch: inference tensors blocking inplace ops in background thread by @albertvillanova in https://github.com/huggingface/trl/pull/5818
• Replace deprecated torch_dtype with dtype across examples, docs, notebooks, tests, and experimental distillation / gold trainers by @qgallouedec in https://github.com/huggingface/trl/pull/5717

Documentation and Examples

• docs(grpo): align model to Qwen2.5 and add GRPO OOM tab in quickstart by @xodn348 in https://github.com/huggingface/trl/pull/5740

CI

• Migrate tests to Qwen3.5 Think/NoThink fixtures + tiny-model generation scripts by @aazizyan in https://github.com/huggingface/trl/pull/5819 and https://github.com/huggingface/trl/pull/5821
• Align tiny Glm4MoeForCausalLM / Cohere / Cohere2 / Qwen2.5-VL configs with their reference models by @qgallouedec in https://github.com/huggingface/trl/pull/5638, https://github.com/huggingface/trl/pull/5706, https://github.com/huggingface/trl/pull/5707 and https://github.com/huggingface/trl/pull/5739
• Fix tiny Qwen3-VL deepstack_visual_indexes and drop the test skip by @qgallouedec in https://github.com/huggingface/trl/pull/5779
• Fix tiny Qwen2.5-VL fullatt_block_indexes out of range for depth=2 by @albertvillanova in https://github.com/huggingface/trl/pull/5805
• Remove non-existent params from tiny Qwen2-VL model by @albertvillanova in https://github.com/huggingface/trl/pull/5795
• Fix vision config num_heads key in Qwen VL tiny model scripts by @matdou in https://github.com/huggingface/trl/pull/5792
• Drop unjustified model.visual. skip in GRPO/RLOO Qwen2.5-VL tests by @qgallouedec in https://github.com/huggingface/trl/pull/5780
• Make the LLaVA / LLaVA-Next test guard explicit by @qgallouedec in https://github.com/huggingface/trl/pull/5778
• Remove obsolete Gemma 3 vision-head guard from VLM training tests by @qgallouedec in https://github.com/huggingface/trl/pull/5772
• Fix OOM in CI: reduce batch size in VLM SFT / GRPO/RLOO VLM / toolcall tests by @albertvillanova in https://github.com/huggingface/trl/pull/5687, https://github.com/huggingface/trl/pull/5767, https://github.com/huggingface/trl/pull/5801
• Fix OOM in CI by clearing chained exception tracebacks by @albertvillanova in https://github.com/huggingface/trl/pull/5776
• Fix OOM in CI by reducing intermediate_size and image token budget for tiny Gemma 4 by @albertvillanova in https://github.com/huggingface/trl/pull/5760
• Fix CI errors in response parsing for gpt-oss/llama with transformers v5 by @albertvillanova in https://github.com/huggingface/trl/pull/5755
• Fix CI AttributeError: 'GptOssConfig' object has no attribute 'num_experts' by @albertvillanova in https://github.com/huggingface/trl/pull/5756
• Fix CI apply_model_revisions by removing _commit_hash kwarg by @albertvillanova in https://github.com/huggingface/trl/pull/5762
• Fix CI test to avoid skipping model.visual params by @albertvillanova in https://github.com/huggingface/trl/pull/5806
• Fix transformers min version for tiny gemma 4 as 5.5.0 by @albertvillanova in https://github.com/huggingface/trl/pull/5763
• Hotfix CI: pin torch < 2.12.0 (later reverted) by @albertvillanova in https://github.com/huggingface/trl/pull/5769
• Fix catch-all empty string in Makefile pytest --only-rerun by @albertvillanova in https://github.com/huggingface/trl/pull/5784
• chore: update tests_latest.yml by @hf-security-analysis[bot] in https://github.com/huggingface/trl/pull/5733

New Contributors

• @hf-security-analysis[bot] made their first contribution in https://github.com/huggingface/trl/pull/5733
• @Beichen-Ma made their first contribution in https://github.com/huggingface/trl/pull/5592
• @DagaBhai made their first contribution in https://github.com/huggingface/trl/pull/5746
• @xodn348 made their first contribution in https://github.com/huggingface/trl/pull/5740
• @mlarnouhet made their first contribution in https://github.com/huggingface/trl/pull/5691
• @matdou made their first contribution in https://github.com/huggingface/trl/pull/5792
• @jamie-peterson-ml made their first contribution in https://github.com/huggingface/trl/pull/5799
• @rycerzes made their first contribution in https://github.com/huggingface/trl/pull/5729

What's Changed

• ⬆️ Bump dev version by @qgallouedec in https://github.com/huggingface/trl/pull/5734
• chore: update tests_latest.yml by @hf-security-analysis[bot] in https://github.com/huggingface/trl/pull/5733
• fix: CUDA memory leak / release BNB dequantization buffers & stale state in OffloadActivations by @butterwecksolutions in https://github.com/huggingface/trl/pull/5730
• fix: invalidate ZeRO-3 param coordinator trace in add_hooks by @roycho96 in https://github.com/huggingface/trl/pull/4693
• Fix nested vocab_size for DistillationTrainer and GOLDTrainer by @Beichen-Ma in https://github.com/huggingface/trl/pull/5592
• feat: add Phi-3.5 training chat templates with generation markers by @DagaBhai in https://github.com/huggingface/trl/pull/5746
• docs(grpo): align model to Qwen2.5 and add GRPO OOM tab in quickstart by @xodn348 in https://github.com/huggingface/trl/pull/5740
• torch_dtype -> dtype by @qgallouedec in https://github.com/huggingface/trl/pull/5717
• Add OpenReward example to the list of examples by @sergiopaniego in https://github.com/huggingface/trl/pull/5752
• Fix CI errors in response parsing for gptoss/llama with transformers v5 by @albertvillanova in https://github.com/huggingface/trl/pull/5755
• Add DDP-2 members to invariant test suite by @qgallouedec in https://github.com/huggingface/trl/pull/5736
• Hotfix CI param not updated AssertionError: Pin torch < 2.12.0 by @albertvillanova in https://github.com/huggingface/trl/pull/5769
• Align tiny-Glm4MoeForCausalLM with GLM-4.5 reference config by @qgallouedec in https://github.com/huggingface/trl/pull/5638
• Align tiny Cohere config with aya-expanse-8b by @qgallouedec in https://github.com/huggingface/trl/pull/5706
• Align tiny Cohere2 config with tiny-aya-earth by @qgallouedec in https://github.com/huggingface/trl/pull/5707
• Fix OOM in CI by reducing intermediate_size and image token budget for tiny Gemma4 by @albertvillanova in https://github.com/huggingface/trl/pull/5760
• Fix CI AttributeError: 'GptOssConfig' object has no attribute 'num_experts' by @albertvillanova in https://github.com/huggingface/trl/pull/5756
• Fix CI apply_model_revisions by removing _commit_hash kwarg by @albertvillanova in https://github.com/huggingface/trl/pull/5762
• Remove obsolete Gemma3 vision-head guard from VLM training tests by @qgallouedec in https://github.com/huggingface/trl/pull/5772
• Replace uv installation script with setup action by @qgallouedec in https://github.com/huggingface/trl/pull/5735
• Fix OOM in CI by clearing chained exception tracebacks by @albertvillanova in https://github.com/huggingface/trl/pull/5776
• Fix transformers min version for tiny gemma4 as 5.5.0 by @albertvillanova in https://github.com/huggingface/trl/pull/5763
• Final logits softcapping support for async GRPO Trainer by @mlarnouhet in https://github.com/huggingface/trl/pull/5691
• Fix vision config num_heads key in Qwen VL tiny model scripts and revert torch pin by @matdou in https://github.com/huggingface/trl/pull/5792
• Drop unjustified model.visual. skip in GRPO / RLOO Qwen2.5-VL tests by @qgallouedec in https://github.com/huggingface/trl/pull/5780
• Fix OOM in CI by reducing batch size and sequence length for toolcall tests by @albertvillanova in https://github.com/huggingface/trl/pull/5801
• Fix exponential backtracking in qwen3 / qwen3_5 / glm4moe response parsing by @xodn348 in https://github.com/huggingface/trl/pull/5798
• Add telemetry to trainers by @qgallouedec in https://github.com/huggingface/trl/pull/5758
• Add Qwen3-VL training chat template with generation markers by @aazizyan in https://github.com/huggingface/trl/pull/5764
• Align tiny Qwen2.5-VL with Qwen/Qwen2.5-VL-3B-Instruct by @qgallouedec in https://github.com/huggingface/trl/pull/5739
• Fix tiny Qwen3-VL deepstack_visual_indexes and drop the test skip by @qgallouedec in https://github.com/huggingface/trl/pull/5779
• Fix OOM in CI by reducing batch size in GRPO/RLOO VLM tests by @albertvillanova in https://github.com/huggingface/trl/pull/5767
• Fix catch-all empty string in Makefile pytest --only-rerun by @albertvillanova in https://github.com/huggingface/trl/pull/5784
• Remove non-existent params from tiny Qwen2-VL model by @albertvillanova in https://github.com/huggingface/trl/pull/5795
• Fix tiny Qwen2.5-VL fullatt_block_indexes out of range for depth=2 by @albertvillanova in https://github.com/huggingface/trl/pull/5805
• Make the LLaVA / LLaVA-Next test guard explicit by @qgallouedec in https://github.com/huggingface/trl/pull/5778
• Fix MPS support in experimental empty_cache() by @jamie-peterson-ml in https://github.com/huggingface/trl/pull/5799
• Fix CI test to avoid skipping model.visual params by @albertvillanova in https://github.com/huggingface/trl/pull/5806
• Align KTO with DPO: Align compute_loss flow by @albertvillanova in https://github.com/huggingface/trl/pull/5810
• Fix generate_batch: inference tensors block inplace ops in background thread by @albertvillanova in https://github.com/huggingface/trl/pull/5818
• Fix metric_for_best_model for trainer-specific eval metrics by @qgallouedec in https://github.com/huggingface/trl/pull/5811
• Align and simplify the stable training scripts by @qgallouedec in https://github.com/huggingface/trl/pull/5812
• Align KTO with DPO: Align _compute_loss_liger flow by @albertvillanova in https://github.com/huggingface/trl/pull/5816
• Add tiny Qwen3.5 Think/NoThink fixture generation scripts by @aazizyan in https://github.com/huggingface/trl/pull/5819
• Migrate tests to Qwen3.5 Think/NoThink fixtures by @aazizyan in https://github.com/huggingface/trl/pull/5821
• Fix OpenRewardSpec omitting task‑scoped tools during rollout binding (fixes #5727) by @rycerzes in https://github.com/huggingface/trl/pull/5729
• Add Qwen3.5 Think/NoThink training chat templates with generation markers by @aazizyan in https://github.com/huggingface/trl/pull/5824
• Release: v1.5 by @qgallouedec in https://github.com/huggingface/trl/pull/5835

Full Changelog: https://github.com/huggingface/trl/compare/v1.4.0...v1.5.0

Copy the contents of any repository directly from the Hub to a Bucket using the new "Copy to Bucket" button on repository pages. Powered by Xet server-side transfers, large files are copied instantly, making it possible to transfer terabytes in just a few seconds. You can now copy massive datasets or model checkpoints from the Hub into your Bucket, then mount the Bucket directly in your training jobs or Spaces.

New Model additions

Cohere2Moe

Command A+ is a Mixture-of-Experts (MoE) language model from Cohere that features a hybrid attention pattern combining sliding window and full attention layers. The model incorporates both shared and routed experts and supports a very large context window for processing extensive text sequences.

Links: Documentation

• Add new cohere2_moe model (#46115) by @Cyrilvallez in #46115

Parakeet tdt (#44171)

• Parakeet tdt (#44171) by @lmaksym

HRM-Text

HRM-Text is an improved autoregressive language-modeling variant of the Hierarchical Reasoning Model (HRM) that uses a hierarchical recurrent forward pass with two transformer stacks - one for slow, abstract planning (H) and one for fast, detailed computation (L) - reused inside a nested recurrence. It features PrefixLM attention where instruction tokens attend bidirectionally while response tokens attend causally, per-head sigmoid output gates, and parameterless RMSNorm. The model is designed as a base language model without instruction tuning or chat templates.

Links: Documentation | Paper

• Add hrm text (#46025) by @abcd1927 in #46025

Breaking changes

The text_embeds input for SAM3, EdgeTAM, and SAM3-Lite-Text models now expects full text embeddings instead of just pooler outputs, aligning with other models in the library — users must update their inputs accordingly.

• 🚨Fix memory leaks caused by lru decorators in vision models (#45922) by @yonigozlan

Audio

Audio support was expanded with the addition of AudioFlamingoNext model checkpoints and improved compilability of audio/vision encoders via standalone pure functions. Additional improvements include better error messaging when loading audio from video files and new documentation for audio/video processors.

• user friendly error when loading audio from video (#45221) by @eustlb in [#45221]
• [docs] adding audio/video processors (#45795) by @stevhliu in [#45795]
• Support Audio Flamingo Next checkpoints (#44830) by @lashahub in [#44830]
• Extract dynamic vision/audio tensors into standalone pure functions (#45396) by @IlyasMoutawwakil in [#45396]

Generation

Fixed generation issues including inputs_embeds and per_layer_inputs handling for Gemma4, an AttributeError in RAG's generate() caused by missing config fields, and flaky VLM generation tests by blocking special image tokens during sampling.

• Fix Gemma4 generation from inputs_embeds and per_layer_inputs (#46049) by @Cyrilvallez in [#46049]
• Fix AttributeError in RAG generate() for missing config fields (#46035) by @Sriniketh24 in [#46035]
• Block image_start/end_token_id in generation test sampling (#45914) by @Rocketknight1 in [#45914]

Bugfixes and improvements

• Remove mask visualization tool from masking_utils.py (#46066) by @Cyrilvallez in [#46066]
• fix: owned_by field in GET /v1/models returns list instead of string (#46006) by @nileshpatil6 in [#46006]
• [CB] Remove OpenTelemetry (#45984) by @remi-or in [#45984]
• docs(readme): use canonical huggingface.co domain in prose links (#46042) by @kiwigitops in [#46042]
• Fix remaining RAG doc examples that crash on current transformers (#46044) by @Sriniketh24 in [#46044]
• Init the actual tensor, not a copy (#46030) by @Rocketknight1 in [#46030]
• docs: sync legacy ACL anthology URLs and update metrics across i18n READMEs (#46027) by @irfaan101 in [#46027]
• [MultimodalLM] add language_model to the get/set_input_embeddings logic (#46029) by @eustlb in [#46029]
• [HRM Text] Add integration tests (#46033) by @vasqu in [#46033]
• hy_v3: add XPU expectations (#45858) by @kaixuanliu in [#45858]
• exaone4_5: add XPU expectations (#45890) by @kaixuanliu in [#45890]
• hyperclovax: add XPU Expectations for CI test (#45926) by @kaixuanliu in [#45926]
• chore(ci): remove dead env vars from circleci-failure-summary-comment.yml (#45972) by @XciD in [#45972]
• [CB] [Major] Add tensor paralellism (#45821) by @remi-or in [#45821]
• docs: update models architecture count and sync ACL anthology URLs (#46001) by @irfaan101 in [#46001]
• bugfix(ci): avoid E2BIG in pr_slow_ci_suggestion (#45983) by @tarekziade in [#45983]
• RFDetr - use correct Roboflow org for release (#45946) by @sbucaille in [#45946]
• docs: Fix formatting issues in weightconverter.md (#45988) by @ArjunSrivastava1 in [#45988]
• Fix colqwen2 test (#45981) by @IlyasMoutawwakil in [#45981]
• Fix M-RoPE device mismatch in Qwen3VL family under FSDP2 CPU offload (#45861) by @jamesbraza in [#45861]
• [docs] chat template prefill (#45947) by @stevhliu in [#45947]
• [docs] decode fast path (#45899) by @stevhliu in [#45899]
• fix: restore _attn_implementation and fix request offset in generate_batch() (#45943) by @sergiopaniego in [#45943]
• Expose per_layer_inputs for every Gemma4 variants (#45927) by @Cyrilvallez in [#45927]
• chore: update benchmark_v2.yml (#45966) by @hf-security-analysis[bot] in [#45966]
• fix(ci): set persist-credentials: false on actions/checkout and close remaining template injection findings (#45964) by @XciD in [#45964]
• chore(ci): set default workflow permissions to contents: read (#45961) by @XciD in [#45961]
• fix(ci): remove template injection on pull_request_target workflows (#45956) by @XciD in [#45956]
• chore(ci): pin all GitHub Actions and reusable workflows by SHA (#45955) by @XciD in [#45955]
• [docs] ALMModelTest (#45900) by @stevhliu in [#45900]
• Enhance apply_chat_template to support custom field prefilling (reasoning_content, thinking, etc.) (#45896) by @Mamiglia in [#45896]
• BUGFIX: Support hubert models that don't have conv_pos_batch_norm configured (#45921) by @igordertigor in [#45921]
• Revert 45777 (#45942) by @Rocketknight1 in [#45942]
• pass the otel secrets (#45933) by @tarekziade in [#45933]
• Add initial torch_tpu backend support (#45918) by @tengomucho in [#45918]
• [CB] Hide activation footprint by using the CUDA graph pool (#45911) by @remi-or in [#45911]
• Require input_ids for repetition penalty (#45389) by @ruben-aghayan in [#45389]
• Fix undefined 'input' variable (#45895) by @fullyz in [#45895]
• Fix post processing RF-DETR (#46041) by @yonigozlan (direct commit on v5.9.0)
• [loading] Free up tensors faster inside ConversionOps (#46110) by @Cyrilvallez (direct commit on v5.9.0)
• Add new cohere2_moe model (#46115) by @Cyrilvallez (direct commit on v5.9.0)
• Fix cohere2 tp_plan for release by @Cyrilvallez (direct commit on v5.9.0)
• Release v5.9.0 by @Cyrilvallez (direct commit on v5.9.0)

Significant community contributions

The following contributors have made significant changes to the library over the last release:

• @lmaksym
  • Parakeet tdt (#44171)
• @eustlb
  • user friendly error when loading audio from video (#45221)
  • [MultimodalLM] add language_model to the get/set_input_embeddings logic (#46029)
• @remi-or
  • [CB] Remove OpenTelemetry (#45984)
  • [CB] [Major] Add tensor paralellism (#45821)
  • [CB] Hide activation footprint by using the CUDA graph pool (#45911)
• @abcd1927
  • Add hrm text (#46025)

Filter dataset benchmark leaderboards by the number of model parameters. On any dataset leaderboard, pick a size range and the rankings refresh to that bucket. The top 3 models in each size category are marked with a 🏅.

🌍 Pick a region when creating buckets and repos

create_bucket and create_repo now accept an optional region argument ("us" or "eu") so you can pin a new bucket or repo to a specific cloud region at creation time. The same option is exposed on the CLI via a --region flag on hf buckets create and hf repos create.

>>> from huggingface_hub import create_bucket, create_repo
>>> create_bucket("my-bucket", region="us")
>>> create_repo("my-model", region="eu")

$ hf buckets create my-bucket --region us
$ hf repos create username/my-model --region eu

• [Bucket/Repo] Support 'region' option in create_bucket and create_repo by @Wauplin in #4194

🧩 Discover marketplace skills with hf skills list

A new hf skills list (alias ls) command lists every skill available in the Hugging Face marketplace and shows whether each one is already installed in the four supported locations (project, global, project Claude, global Claude). Handy when you want to check what's installable and what you've already got before running hf skills add.

$ hf skills ls
NAME                        DESCRIPTION                         PROJECT PROJECT (CLAUDE) GLOBAL GLOBAL (CLAUDE)
--------------------------- ----------------------------------- ------- ---------------- ------ ---------------
hf-cli                      Execute Hugging Face Hub operati...     yes              yes    yes             yes                                

• [CLI] Add hf skills list command by @Wauplin in #4180

🎨 Polished --help output with ANSI styling

hf --help and every subcommand now render with underlined section headings and bold option/command names, making the help screens much easier to scan in a terminal. The new styling is automatically disabled when NO_COLOR is set or when the CLI detects it's running under an AI agent, so script and agent output stays clean.

• [CLI] Pretty-print --help with ANSI styling by @Wauplin in #4192

🖥️ CLI

• [CLI] Check Homebrew registry for updates when installed via brew by @Wauplin in #4204 — hf update no longer suggests a version that isn't on brew yet for Homebrew installs.
• [CLI] No traceback on LocalEntryNotFoundError by @Wauplin in #4190 — offline/cache-miss errors now print a clean message instead of a Python traceback (set HF_DEBUG=1 for the full stack).

🐛 Bug and typo fixes

• Make HF_HUB_ENABLE_HF_TRANSFER deprecation warning visible to users by @Adithya191101 in #4220
• Fix hint message to use 'hf skills update' by @Pierrci in #4206

📖 Documentation

• [docs] Drop duplicated Key Features list from hf jobs CLI section by @davanstrien in #4222

🏗️ Internal

• [CI] Harden style-bot workflow against TOCTOU by @paulinebm in #4183
• Only sync skill if SKILL.md has changed by @evalstate in #4210

What's Changed

• Bump dev version: v1.25.0.dev0 by @albertvillanova in https://github.com/huggingface/smolagents/pull/1939
• Fix import of TokenUsage by @albertvillanova in https://github.com/huggingface/smolagents/pull/1943
• Add MLflow integration doc by @B-Step62 in https://github.com/huggingface/smolagents/pull/1884
• Refactor agent/model deserialization: Replace importlib with registry pattern by @albertvillanova in https://github.com/huggingface/smolagents/pull/1944
• Update _function_type_hints_utils.py: CohereForAI -> CohereLabs by @Michellehbn in https://github.com/huggingface/smolagents/pull/1946
• Fix High impact vulnerability on remote executors by @nnfrog in https://github.com/huggingface/smolagents/pull/1637
• Fix SafeSerializer.get_deserializer_code by @albertvillanova in https://github.com/huggingface/smolagents/pull/1983
• Fix RemotePythonExecutor._deserialize_final_answer for legacy no-prefix pickle by @albertvillanova in https://github.com/huggingface/smolagents/pull/1984
• Test 'pickle:...' payloads are properly decoded by @albertvillanova in https://github.com/huggingface/smolagents/pull/1985
• Remove support for legacy no-prefix pickle by @albertvillanova in https://github.com/huggingface/smolagents/pull/1986
• Create docs reference page for Python executors by @albertvillanova in https://github.com/huggingface/smolagents/pull/1987
• Isolate WasmExecutor Deno cache to per-instance temp directory by @albertvillanova in https://github.com/huggingface/smolagents/pull/1995
• Switch wildcard bind to loopback-only endpoint in WasmExecutor by @albertvillanova in https://github.com/huggingface/smolagents/pull/2025
• Use token in DockerExecutor by @albertvillanova in https://github.com/huggingface/smolagents/pull/2027
• Remove allow_origin from Docker and Modal executors by @albertvillanova in https://github.com/huggingface/smolagents/pull/2028
• Use token in WasmExecutor by @albertvillanova in https://github.com/huggingface/smolagents/pull/2031
• Fix evaluate_with() calling exit on wrong object by @dubin555 in https://github.com/huggingface/smolagents/pull/2029
• Fix evaluate_with by suppressing exception if truthy exit by @albertvillanova in https://github.com/huggingface/smolagents/pull/2033
• Use requests.Session in WasmExecutor by @albertvillanova in https://github.com/huggingface/smolagents/pull/2032
• Fix Rich MarkupError in AgentLogger.log_task by rendering untrusted text safely by @Barcavin in https://github.com/huggingface/smolagents/pull/1894
• Emphasise sandboxing is required; LocalPythonExecutor is not a security tool by @zsims in https://github.com/huggingface/smolagents/pull/2039
• Fix log_error rendering brackets as Rich markup by @albertvillanova in https://github.com/huggingface/smolagents/pull/2077
• Support huggingface-hub >= 1 by @merveenoyan in https://github.com/huggingface/smolagents/pull/2075
• Remove unused safe_serialization param from _websocket_run_code_raise_errors by @albertvillanova in https://github.com/huggingface/smolagents/pull/2131
• Set allow_pickle default to False in internal calls by @albertvillanova in https://github.com/huggingface/smolagents/pull/2132
• 🔒 Pin GitHub Actions to commit SHAs by @paulinebm in https://github.com/huggingface/smolagents/pull/2149
• chore: bump doc-builder SHA for PR upload workflow by @rtrompier in https://github.com/huggingface/smolagents/pull/2194
• chore: bump doc-builder SHA for main doc build workflow by @rtrompier in https://github.com/huggingface/smolagents/pull/2219

New Contributors

• @B-Step62 made their first contribution in https://github.com/huggingface/smolagents/pull/1884
• @Michellehbn made their first contribution in https://github.com/huggingface/smolagents/pull/1946
• @dubin555 made their first contribution in https://github.com/huggingface/smolagents/pull/2029
• @zsims made their first contribution in https://github.com/huggingface/smolagents/pull/2039
• @paulinebm made their first contribution in https://github.com/huggingface/smolagents/pull/2149
• @rtrompier made their first contribution in https://github.com/huggingface/smolagents/pull/2194

Full Changelog: https://github.com/huggingface/smolagents/compare/v1.24.0...v1.25.0

This release is mainly to fix the Deepseek V4 integration!!!

• [fix] Add fatal_error to ContinuousBatchingManager so the serving... by @qgallouedec, @remi-or
• Fix WeightConverter regex incorrectly matching shared_experts as experts by @silencelamb, @claude
• Fix deepseek v4 by @ArthurZucker (#45892)
• Deepseek v4 csa mask collapse by @ArthurZucker, @Sawyer117 (#45928)

Features

Chunked cross-entropy loss for SFT (up to –50% VRAM)

A new loss_type="chunked_nll" option drastically reduces peak activation memory in SFT by avoiding the full [batch × seq × vocab] logits tensor. Ignored-label tokens are dropped before the lm_head matmul, and the cross-entropy is computed over the remaining tokens in checkpointed chunks (default chunk_size=256, the sweet spot consistent across model sizes and sequence lengths).

from trl import SFTConfig, SFTTrainer

trainer = SFTTrainer(
    model="Qwen/Qwen3-4B",
    args=SFTConfig(loss_type="chunked_nll"),
    train_dataset=dataset,
)
trainer.train()

Peak GPU memory, AdamW fp32:

End-to-end, chunked NLL is consistently as fast or faster than nll — and it unlocks sequence lengths that don't fit at all under the standard path.

The chunked path also supports VLMs (https://github.com/huggingface/trl/pull/5684).

by @qgallouedec in https://github.com/huggingface/trl/pull/5575, https://github.com/huggingface/trl/pull/5676 and https://github.com/huggingface/trl/pull/5684

OpenReward Standard environment adapter (experimental)

A new trl.experimental.openreward adapter plugs any environment speaking the Open Reward Standard (ORS) protocol into any TRL trainer accepting an environment_factory (GRPOTrainer, AsyncGRPOTrainer). One identifier wires all three trainer slots — dataset, factory, reward_func:

from trl import GRPOConfig, GRPOTrainer
from trl.experimental.openreward import OpenRewardEnv

env = OpenRewardEnv("Eigent/SETA")  # or "http://localhost:8000"

trainer = GRPOTrainer(
    model="Qwen/Qwen3-4B",
    args=GRPOConfig(...),
    train_dataset=env.dataset,
    environment_factory=env.factory,
    reward_funcs=env.reward_func,
)

Tools are bound dynamically from JSON Schema at construction (no per-env wrapper code), and env.dataset autoderives task lists from the ORS task endpoints. The same code path works for envs hosted on the OpenReward platform, self-hosted on any container service, or running locally on localhost. A SETA training example is included.

by @adithya-s-k in https://github.com/huggingface/trl/pull/5696

Training-invariance test suite

Unit tests don't catch trainer-level numerical drift (gradient-accumulation normalization bugs, attention-impl divergence (eager ↔ FA2 / kernels)) they silently shift the loss trajectory and users only notice when their run no longer reproduces. (Cf. last year's transformers grad-accum bug, or the "We found two bugs in DeepSpeed" paper.)

A new opt-in pytest -m invariant suite asserts the loss / grad_norm trajectory of short end-to-end SFT/DPO runs against committed reference snapshots, with equivalence classes for configs that should produce identical trajectories (e.g. pdb=1, gas=8 ≡ default; eager ≡ FA2 ≡ kernels). Hardware-pinned to H100 80GB, real pretrained model, full_determinism, fixed seed. Initial coverage: 2 trainers × 2 invariance axes (grad-accum, attn-impl) × gradient-checkpointing equivalence.

by @qgallouedec in https://github.com/huggingface/trl/pull/5686, https://github.com/huggingface/trl/pull/5688 and https://github.com/huggingface/trl/pull/5689

MFU helpers

Three new pure helpers in trl.trainer.utils for measuring training efficiency:

• compute_flops_per_token(config, seq_len) — handles dense and MoE (Mixtral, Qwen3-MoE, DeepSeek-V2)
• compute_mfu(flops_per_token, tps, world_size, peak_flops) — Model FLOPs Utilization as a percentage
• adjusted_mfu(mfu, config, seq_len) — non-causal → causal-corrected (Llama / DS Ulysses convention)

by @AmineDiro in https://github.com/huggingface/trl/pull/5698

GRPO Liger kernel update (Liger 0.8.0)

GRPO's Liger-kernel integration is updated for Liger 0.8.0: delta two-sided clipping, use_bias_correction_kl, and SAPO/VESPO parameters are now forwarded into LigerFusedLinearGRPOLoss. The previous delta + use_liger_kernel guard is removed — both can be combined.

by @kashif in https://github.com/huggingface/trl/pull/5690

Length-normalized DPO sigmoid loss

A new loss_type="sigmoid_norm" option for DPOConfig implements the per-token (length-normalized) DPO loss used by Tülu 3 / OLMo (paper §5.1.2 eq. 6) to mitigate length bias.

from trl import DPOConfig, DPOTrainer

trainer = DPOTrainer(
    model="Qwen/Qwen3-4B",
    args=DPOConfig(loss_type="sigmoid_norm"),
    train_dataset=dataset,
)

by @BrownianNotion in https://github.com/huggingface/trl/pull/5406

Even more training chat templates

Four more model families gain training-compatible chat templates with {% generation %} markers (assistant-only loss masking) and/or response schemas (tool-calling parsing):

• Cohere training template by @dschulmeist in https://github.com/huggingface/trl/pull/5627
• Cohere2 {% generation %} markers by @qgallouedec in https://github.com/huggingface/trl/pull/5675
• Gemma 3 training template by @hwanython in https://github.com/huggingface/trl/pull/5685
• Qwen3-2507 training template by @SwayamInSync in https://github.com/huggingface/trl/pull/5574
• Qwen2.5 response schema by @aazizyan in https://github.com/huggingface/trl/pull/5728

get_training_chat_template now also accepts a processor (not just a tokenizer) — useful for VLMs (https://github.com/huggingface/trl/pull/5560).

KTO ↔ DPO alignment: closing in on graduation

Another batch of alignment PRs this cycle. KTO and DPO are now structurally aligned across PEFT handling, model initialization, training-arg grouping, ref-logp precomputation, and metric handling — promotion of KTO out of experimental is imminent.

PRs (all by @albertvillanova): #5659, #5660, #5661, #5679, #5701, #5702, #5703, #5704, #5705, #5714.

Other

• Reject parallelism_config with cp_size>1 or sp_size>1 in GRPO/RLOO — fail fast at config init with a clear error instead of mid-training crash. By @kashif in https://github.com/huggingface/trl/pull/5699
• Fail early for unsupported PEFT + Liger Kernel in DPO by @albertvillanova in https://github.com/huggingface/trl/pull/5709
• Explicitly set model_accepts_loss_kwargs=False in DPO and Reward by @albertvillanova in https://github.com/huggingface/trl/pull/5710
• Set _tokenizer attribute in experimental trainers by @albertvillanova in https://github.com/huggingface/trl/pull/5566
• Simplify peft_config handling in core / experimental trainers by @albertvillanova in https://github.com/huggingface/trl/pull/5673 and https://github.com/huggingface/trl/pull/5674
• Replace isinstance with is_peft_model / drop redundant is_peft_available by @albertvillanova in https://github.com/huggingface/trl/pull/5682 and https://github.com/huggingface/trl/pull/5683
• Reduce inconsistency across trainer test files by @qgallouedec in https://github.com/huggingface/trl/pull/5678
• Refactor tiny-model generation scripts by @qgallouedec in https://github.com/huggingface/trl/pull/5637
• Revert VLM support in parse_response by @qgallouedec in https://github.com/huggingface/trl/pull/5561

Fixes

• 5 GB+ CUDA memory leak in activation offloading — OffloadActivations.__exit__ now syncs the compute/offload streams and clears the stash dictionaries, preventing orphaned offload tensors from leaking onto a dead stream (~0.2 GiB/step accumulation observed during QLoRA vision training before the fix). By @butterwecksolutions in https://github.com/huggingface/trl/pull/5694 and https://github.com/huggingface/trl/pull/5700
• Fix reverse-KL server path NaN on variable completion length in DistillationTrainer by @k1064190 in https://github.com/huggingface/trl/pull/5594
• GKDTrainer: fix return_outputs in the Liger kernel path by @roycho96 in https://github.com/huggingface/trl/pull/4688
• GKDTrainer: fix seq-KD wasted teacher forward by @roycho96 in https://github.com/huggingface/trl/pull/5726
• GKDTrainer: fix Liger fused JSD path computing wrong loss by @roycho96 in https://github.com/huggingface/trl/pull/5731
• Fix missing PEFT validation when passing peft_config to core / experimental trainers by @albertvillanova in https://github.com/huggingface/trl/pull/5664 and https://github.com/huggingface/trl/pull/5665
• Fix peft_config type hint in experimental trainers by @albertvillanova in https://github.com/huggingface/trl/pull/5666
• Fix discarded assertion message in trainer parameter checks by @qgallouedec in https://github.com/huggingface/trl/pull/5677
• Fix typo in model name in README by @qgallouedec in https://github.com/huggingface/trl/pull/5711

Documentation and Examples

• Upload testing suite for DistillationTrainer by @cmpatino in https://github.com/huggingface/trl/pull/5615

CI

• Fix OOM in CI by reducing batch size in VLM SFT tests by @albertvillanova in https://github.com/huggingface/trl/pull/5687
• Fix OOM in CI by reducing image size of tiny Gemma 3 model by @albertvillanova in https://github.com/huggingface/trl/pull/5680
• Fix OOM in CI test reruns due to GPU memory leak from traceback frame locals by @albertvillanova in https://github.com/huggingface/trl/pull/5681
• Add tiny Qwen3-4B-Instruct-2507 by @qgallouedec in https://github.com/huggingface/trl/pull/5586
• Align tiny Qwen3 MoE config with Qwen/Qwen3-30B-A3B by @qgallouedec in https://github.com/huggingface/trl/pull/5716
• Fix GRPO VLM tests: multimodal training requires conversational prompts by @kaixuanliu in https://github.com/huggingface/trl/pull/5550
• Regenerate invariance data + relax the tolerance by @qgallouedec in https://github.com/huggingface/trl/pull/5688

New Contributors

• @dschulmeist made their first contribution in https://github.com/huggingface/trl/pull/5627
• @k1064190 made their first contribution in https://github.com/huggingface/trl/pull/5594
• @butterwecksolutions made their first contribution in https://github.com/huggingface/trl/pull/5694
• @hwanython made their first contribution in https://github.com/huggingface/trl/pull/5685
• @BrownianNotion made their first contribution in https://github.com/huggingface/trl/pull/5406
• @adithya-s-k made their first contribution in https://github.com/huggingface/trl/pull/5696
• @roycho96 made their first contribution in https://github.com/huggingface/trl/pull/4688
• @aazizyan made their first contribution in https://github.com/huggingface/trl/pull/5728

What's Changed

• ⬆️ Bump dev version by @qgallouedec in https://github.com/huggingface/trl/pull/5648
• Align KTO with DPO: Remove model_init parameter by @albertvillanova in https://github.com/huggingface/trl/pull/5659
• Align KTO with DPO: Remove preprocess_logits_for_metrics parameter by @albertvillanova in https://github.com/huggingface/trl/pull/5660
• Add tiny Qwen3-4B-Instruct-2507 by @qgallouedec in https://github.com/huggingface/trl/pull/5586
• Chunked cross-entropy loss for SFT (up to –50% VRAM) by @qgallouedec in https://github.com/huggingface/trl/pull/5575
• Fix missing PEFT validation when passing peft_config to core trainers by @albertvillanova in https://github.com/huggingface/trl/pull/5664
• Fix missing PEFT availability check when passing peft_config to experimental trainers by @albertvillanova in https://github.com/huggingface/trl/pull/5665
• Align KTO with DPO: Align PEFT handling by @albertvillanova in https://github.com/huggingface/trl/pull/5661
• Set _tokenizer attribute in experimental trainers by @albertvillanova in https://github.com/huggingface/trl/pull/5566
• Fix peft_config type hint in experimental trainers by @albertvillanova in https://github.com/huggingface/trl/pull/5666
• Add Cohere training chat template by @dschulmeist in https://github.com/huggingface/trl/pull/5627
• Simplify peft_config handling in core trainers by @albertvillanova in https://github.com/huggingface/trl/pull/5673
• Simplify peft_config handling in experimental trainers by @albertvillanova in https://github.com/huggingface/trl/pull/5674
• fix(distillation): reverse-KL server path NaN on variable completion length by @k1064190 in https://github.com/huggingface/trl/pull/5594
• Fix discarded assertion message in trainer parameter checks by @qgallouedec in https://github.com/huggingface/trl/pull/5677
• Align KTO with DPO: Replace direct type check with is_peft_model by @albertvillanova in https://github.com/huggingface/trl/pull/5679
• Remove redundant is_peft_available from core trainers by @albertvillanova in https://github.com/huggingface/trl/pull/5682
• Replace isinstance with is_peft_model in experimental trainers by @albertvillanova in https://github.com/huggingface/trl/pull/5683
• Upload testing suite for DistillationTrainer by @cmpatino in https://github.com/huggingface/trl/pull/5615
• Fix OOM in CI by reducing batch size in VLM SFT tests by @albertvillanova in https://github.com/huggingface/trl/pull/5687
• Fix OOM in CI by reducing image size of tiny Gemma3 model by @albertvillanova in https://github.com/huggingface/trl/pull/5680
• Fix OOM in CI test reruns due to GPU memory leak from traceback frame locals by @albertvillanova in https://github.com/huggingface/trl/pull/5681
• Add training-invariance tests by @qgallouedec in https://github.com/huggingface/trl/pull/5686
• Regenerate invariance data + relax the tolerance by @qgallouedec in https://github.com/huggingface/trl/pull/5688
• fix: prevent RuntimeError crash in activation offloading for non-contiguous tensors by @butterwecksolutions in https://github.com/huggingface/trl/pull/5694
• [GRPO] update Liger-kernel grpo loss (delta, vespo, KL bias correction) by @kashif in https://github.com/huggingface/trl/pull/5690
• Extend invariant suite with gradient-checkpointing equivalence by @qgallouedec in https://github.com/huggingface/trl/pull/5689
• Add Gemma 3 training chat template by @hwanython in https://github.com/huggingface/trl/pull/5685
• Add {% generation %} markers for Cohere2 chat template by @qgallouedec in https://github.com/huggingface/trl/pull/5675
• Add length-normalized sigmoid loss type to DPO trainer by @BrownianNotion in https://github.com/huggingface/trl/pull/5406
• Add training chat template for Qwen3-2507 by @SwayamInSync in https://github.com/huggingface/trl/pull/5574
• Align KTO with DPO: Remove enforcement of causal language models by @albertvillanova in https://github.com/huggingface/trl/pull/5701
• Align KTO with DPO: Remove duplicate import of PreTrainedModel by @albertvillanova in https://github.com/huggingface/trl/pull/5702
• Align KTO with DPO: Simplify max_length init logic by @albertvillanova in https://github.com/huggingface/trl/pull/5703
• Align KTO with DPO: Group training arguments by @albertvillanova in https://github.com/huggingface/trl/pull/5704
• Align KTO with DPO: Use _metrics attribute by @albertvillanova in https://github.com/huggingface/trl/pull/5705
• Reduce inconsistency across trainer test files by @qgallouedec in https://github.com/huggingface/trl/pull/5678
• Refactor tiny-model generation scripts by @qgallouedec in https://github.com/huggingface/trl/pull/5637
• Accept processor in get_training_chat_template by @qgallouedec in https://github.com/huggingface/trl/pull/5560
• Enable chunked NLL loss with PEFT in SFT by @qgallouedec in https://github.com/huggingface/trl/pull/5676
• Fix GRPO VLM tests: Multimodal training requires conversational prompts by @kaixuanliu in https://github.com/huggingface/trl/pull/5550
• [experimental] Add OpenReward Standard environment adapter by @adithya-s-k in https://github.com/huggingface/trl/pull/5696
• GKDTrainer: Fix return_outputs in Liger kernel path and update tests by @roycho96 in https://github.com/huggingface/trl/pull/4688
• Reject parallelism_config with cp_size>1 or sp_size>1 in GRPO/RLOO by @kashif in https://github.com/huggingface/trl/pull/5699
• Fix typo in model name in README by @qgallouedec in https://github.com/huggingface/trl/pull/5711
• Explicitly set model_accepts_loss_kwargs=False in DPO and Reward by @albertvillanova in https://github.com/huggingface/trl/pull/5710
• Fail early for unsupported PEFT + Liger Kernel in DPO by @albertvillanova in https://github.com/huggingface/trl/pull/5709
• Revert VLM support in parse_response by @qgallouedec in https://github.com/huggingface/trl/pull/5561
• Align KTO with DPO: Align _precompute_ref_logps by @albertvillanova in https://github.com/huggingface/trl/pull/5714
• fix: prevent 5 GB+ CUDA memory leak in activation offloading by syncing streams and clear stashes in OffloadActivations.exit by @butterwecksolutions in https://github.com/huggingface/trl/pull/5700
• Align tiny Qwen3 MoE config with Qwen/Qwen3-30B-A3B by @qgallouedec in https://github.com/huggingface/trl/pull/5716
• Add MFU helpers by @AmineDiro in https://github.com/huggingface/trl/pull/5698
• [GKD] Fix seq kd wasted teacher forward by @roycho96 in https://github.com/huggingface/trl/pull/5726
• Add Qwen2.5 response schema by @aazizyan in https://github.com/huggingface/trl/pull/5728
• Enable chunked NLL loss with VLM in SFT by @qgallouedec in https://github.com/huggingface/trl/pull/5684
• [GKD] Fix Liger fused JSD path computing wrong loss by @roycho96 in https://github.com/huggingface/trl/pull/5731
• Release: v1.4 by @qgallouedec in https://github.com/huggingface/trl/pull/5732

Full Changelog: https://github.com/huggingface/trl/compare/v1.3.0...v1.4.0

April 23, 2026

• Add Gemma4 ViT encoders w/ NaFlex pipeline support (variable aspect/size per image). Thanks Yonghye Kwon
• Support DINOv3 weights in NaFlexVit. Thanks Yonghye Kwon
• Some improvements to Muon fallback (AdamW/NadamW) lr behavior

What's Changed

• 🔒 Pin GitHub Actions to commit SHAs by @paulinebm in https://github.com/huggingface/pytorch-image-models/pull/2689
• Improve fallback (adamw/nadamw) LR handling for Muon optimizer by @rwightman in https://github.com/huggingface/pytorch-image-models/pull/2688
• Fix NaFlexVit DINOv3 support: propagate rope_type and rotate_half by @developer0hye in https://github.com/huggingface/pytorch-image-models/pull/2692
• chore: bump doc-builder SHA for PR upload workflow by @rtrompier in https://github.com/huggingface/pytorch-image-models/pull/2694
• Gemma4 by @rwightman in https://github.com/huggingface/pytorch-image-models/pull/2697
• Add encoder_pool option to gemma4 classification model to toggle soft… by @rwightman in https://github.com/huggingface/pytorch-image-models/pull/2698
• Fix some performance regressions with torch.compile + Tasks. Fix #2693 by @rwightman in https://github.com/huggingface/pytorch-image-models/pull/2699

New Contributors

• @paulinebm made their first contribution in https://github.com/huggingface/pytorch-image-models/pull/2689

Full Changelog: https://github.com/huggingface/pytorch-image-models/compare/v1.0.26...v1.0.27