v0.18.0 — TRL — releases.sh

Major or breaking

PEFT support for Liger GRPO by @SalmanMohammadi in https://github.com/huggingface/trl/pull/3355
[🐯+GRPO] Support FSDP + Fix bug when using LigerGRPO with DDP by @shivam15s in https://github.com/huggingface/trl/pull/3260
🤝 Compatibility of the TRL CLI with accelerate arguments by @qgallouedec in https://github.com/huggingface/trl/pull/3409
🧑‍🤝‍🧑 Co-Locating vLLM w/ training to for higher throughput and GPU utilization by @toslali-ibm in https://github.com/huggingface/trl/pull/3394
✌️ Add support for FSDP2 by @lewtun in https://github.com/huggingface/trl/pull/3317
💔 [GRPO] Decouple gradient accumulation from the number of minibatches generated by @edbeeching in https://github.com/huggingface/trl/pull/3388
[Models] Activation checkpointing from TorchTune by @kashif in https://github.com/huggingface/trl/pull/2954
feat: Implement Two-Sided Clipping for GRPO Trainer by @ucalyptus in https://github.com/huggingface/trl/pull/3434
🎁 Reward submodule by @qgallouedec in https://github.com/huggingface/trl/pull/3430

What's Changed

⬆️ Bump dev version by @qgallouedec in https://github.com/huggingface/trl/pull/3357
🔢 Pad to multiple of by @qgallouedec in https://github.com/huggingface/trl/pull/3362
🥸🔢 Adding pad_multiple to SFT trainer by @shirinyamani in https://github.com/huggingface/trl/pull/3365
🎭 Fix train and eval mode checking in GRPOTrainer and SFTTrainer by @I-l-l-I in https://github.com/huggingface/trl/pull/3337
Better guards for DeepSpeed imports by @lewtun in https://github.com/huggingface/trl/pull/3351
⚰️ Remove deprecated by @qgallouedec in https://github.com/huggingface/trl/pull/3364
📋 Allow calling trl cli in sft mode with config file by @CloseChoice in https://github.com/huggingface/trl/pull/3380
PEFT support for Liger GRPO by @SalmanMohammadi in https://github.com/huggingface/trl/pull/3355
DPO fixes for evaluations by @winglian in https://github.com/huggingface/trl/pull/3377
Deprecate TextEnvironment and tools by @lewtun in https://github.com/huggingface/trl/pull/3389
[🐯+GRPO] Support FSDP + Fix bug when using LigerGRPO with DDP by @shivam15s in https://github.com/huggingface/trl/pull/3260
[GRPO] Reference model initialization bug fix by @LeonEricsson in https://github.com/huggingface/trl/pull/3397
🌊 Add MLflow metrics in profiling context by @dhruvmullick in https://github.com/huggingface/trl/pull/3400
🧑‍🤝‍🧑 Co-Locating vLLM w/ training to for higher throughput and GPU utilization by @toslali-ibm in https://github.com/huggingface/trl/pull/3394
✨ [IterativeSFT] Small refresher by @LeonEricsson in https://github.com/huggingface/trl/pull/3378
💔 [SFT] Raise error when formatting_func is used with completion_only_loss by @LeonEricsson in https://github.com/huggingface/trl/pull/3385
🦁 Fix liger initialization by @shivam15s in https://github.com/huggingface/trl/pull/3401
👉 [DPO] Model forward pass padding side fix by @LeonEricsson in https://github.com/huggingface/trl/pull/3307
🪪 Remove license classifier by @qgallouedec in https://github.com/huggingface/trl/pull/3402
🕺 Migrate setup configuration from setup.py to setup.cfg and make rich an optional dep by @qgallouedec in https://github.com/huggingface/trl/pull/3403
🕊️ Un-restrict diffusers by @qgallouedec in https://github.com/huggingface/trl/pull/3407
✌️ Add support for FSDP2 by @lewtun in https://github.com/huggingface/trl/pull/3317
🤝 Compatibility of the TRL CLI with accelerate arguments by @qgallouedec in https://github.com/huggingface/trl/pull/3409
💔 [GRPO] Decouple gradient accumulation from the number of minibatches generated by @edbeeching in https://github.com/huggingface/trl/pull/3388
🎲 [GRPO] Shuffle mini batches by @edbeeching in https://github.com/huggingface/trl/pull/3391
📝 vLLM-integration documentation by @shirinyamani in https://github.com/huggingface/trl/pull/3376
🎁 Reward takes completion ids by @qgallouedec in https://github.com/huggingface/trl/pull/3272
🐍 Support Python 3.13 by @qgallouedec in https://github.com/huggingface/trl/pull/2593
[Models] Activation checkpointing from TorchTune by @kashif in https://github.com/huggingface/trl/pull/2954
🧪 Testing support for Qwen3 tiny by @shirinyamani in https://github.com/huggingface/trl/pull/3415
Update README.md by @qgallouedec in https://github.com/huggingface/trl/pull/3420
🏹 Support kv_cache_dtype to quantize kv-cache in vllm by @winglian in https://github.com/huggingface/trl/pull/3422
enable trl env on xpu by @yao-matrix in https://github.com/huggingface/trl/pull/3438
use device agnostic empty_cache in ppo & rloo by @yao-matrix in https://github.com/huggingface/trl/pull/3439
feat: Implement Two-Sided Clipping for GRPO Trainer by @ucalyptus in https://github.com/huggingface/trl/pull/3434
🎁 Reward submodule by @qgallouedec in https://github.com/huggingface/trl/pull/3430
[CI] fix CI failure of transformer dev by @kashif in https://github.com/huggingface/trl/pull/3457
enable vllm c-s tests on XPU by @yao-matrix in https://github.com/huggingface/trl/pull/3445
enable activation offloading on XPU by @yao-matrix in https://github.com/huggingface/trl/pull/3444
🙅 PPO value_model can't be None, so it shouldn't be Optional by @AMindToThink in https://github.com/huggingface/trl/pull/3300
[NashMD] fix the edge case where the model is a peft model by @kashif in https://github.com/huggingface/trl/pull/3473
Update .pre-commit-config.yaml by @kashif in https://github.com/huggingface/trl/pull/3479
[SFT] update minimal liger version by @kashif in https://github.com/huggingface/trl/pull/3483
[CI] fix sampler api to make the CI green by @kashif in https://github.com/huggingface/trl/pull/3488
Fix typo by @nikolai-kummer in https://github.com/huggingface/trl/pull/3489
[Doc][SFT] Update sft_trainer.md. link prompt-completion dataset example by @HERIUN in https://github.com/huggingface/trl/pull/3486
Fix mis-aligned prompts and completions in colocate mode by @toslali-ibm in https://github.com/huggingface/trl/pull/3491
[Docs] sync logging doc to current metrics by @kashif in https://github.com/huggingface/trl/pull/3478
[GRPO] disabling top_k sampling default by @kashif in https://github.com/huggingface/trl/pull/3494
[GKD] fix the gkd script by @kashif in https://github.com/huggingface/trl/pull/3497
👇 Update grpo.py to fix bugs for cli grpo --reward_funcs my_lib.my_reward by @wa008 in https://github.com/huggingface/trl/pull/3454
🛠️ Initialize reward_kwargs to prevent UnboundLocalError in GRPOTrainer by @teilomillet in https://github.com/huggingface/trl/pull/3459
🐌 Clean two-sided clipping by @qgallouedec in https://github.com/huggingface/trl/pull/3499
🔭 [GRPO] Log advantages and fraction of samples with an std of zero by @edbeeching in https://github.com/huggingface/trl/pull/3502
📏 Completion length logging fix + remainder logging fix by @shirinyamani in https://github.com/huggingface/trl/pull/3482
🤧 LD-DPO support by @AIR-hl in https://github.com/huggingface/trl/pull/3458
🏰 [vllm] Support base_url parameter for vLLM client initialization by @re-imagined in https://github.com/huggingface/trl/pull/3324
✂️ [DPO] Fix truncation keep_end leading to zero'd out samples by @LeonEricsson in https://github.com/huggingface/trl/pull/3398
Release: v0.18 by @qgallouedec in https://github.com/huggingface/trl/pull/3504

New Contributors

@CloseChoice made their first contribution in https://github.com/huggingface/trl/pull/3380
@SalmanMohammadi made their first contribution in https://github.com/huggingface/trl/pull/3355
@dhruvmullick made their first contribution in https://github.com/huggingface/trl/pull/3400
@toslali-ibm made their first contribution in https://github.com/huggingface/trl/pull/3394
@yao-matrix made their first contribution in https://github.com/huggingface/trl/pull/3438
@nikolai-kummer made their first contribution in https://github.com/huggingface/trl/pull/3489
@wa008 made their first contribution in https://github.com/huggingface/trl/pull/3454
@teilomillet made their first contribution in https://github.com/huggingface/trl/pull/3459
@re-imagined made their first contribution in https://github.com/huggingface/trl/pull/3324

Full Changelog: https://github.com/huggingface/trl/compare/v0.17.0...v0.18.0