v0.15.0 — TRL — releases.sh

Major and breaking changes

Coming soon

What's Changed

⬆️ Bump dev version by @qgallouedec in https://github.com/huggingface/trl/pull/2689
📦 trl.templates in excluded packages by @qgallouedec in https://github.com/huggingface/trl/pull/2690
📖 Docs fix spelling issues by @nnsW3 in https://github.com/huggingface/trl/pull/2682
📄 Add GRPO batch size note in docs by @sdpkjc in https://github.com/huggingface/trl/pull/2672
🙈 Fixed typo in the GRPO documentation by @famouswizard in https://github.com/huggingface/trl/pull/2691
docs: Fix broken "Good First Issue" link in CONTRIBUTING.md by @famouswizard in https://github.com/huggingface/trl/pull/2693
🧠 Fix typo in "understand" in ppo_trainer.md by @famouswizard in https://github.com/huggingface/trl/pull/2695
☠️ Remove deprecated by @qgallouedec in https://github.com/huggingface/trl/pull/2692
💡 Add "Mini-R1: Reproduce Deepseek R1 „aha moment“ a RL tutorial" by @qgallouedec in https://github.com/huggingface/trl/pull/2697
📋 Add eval loss logging during prediction in GRPO by @kashif in https://github.com/huggingface/trl/pull/2694
fix: Fix typo in filename Update ultrafeedback.py by @brawncode in https://github.com/huggingface/trl/pull/2699
📖 Add GRPOTrainer to README.md by @burtenshaw in https://github.com/huggingface/trl/pull/2713
Improve GRPO example by @lewtun in https://github.com/huggingface/trl/pull/2717
📖 Nit Fix in Documentation by @ParagEkbote in https://github.com/huggingface/trl/pull/2722
🏰 num_logits_to_keep to logits_to_keep by @qgallouedec in https://github.com/huggingface/trl/pull/2721
💰 Fix incorrect calculation in Olivia's baguette spending logic by @defiberrys in https://github.com/huggingface/trl/pull/2727
fix: Fix typo in filename in ultrafeedback-prompt.py by @brawncode in https://github.com/huggingface/trl/pull/2716
docs: Fix typos in alias descriptions by @defiberrys in https://github.com/huggingface/trl/pull/2729
⚠️ Fix Attention Masking in GRPO by @andyl98 in https://github.com/huggingface/trl/pull/2708
🔂 Use vLLM prefix caching for speedup by @winglian in https://github.com/huggingface/trl/pull/2757
💔 Decouple loss computing and generation in GRPO by @qgallouedec in https://github.com/huggingface/trl/pull/2762
📌 vLLM >= 0.7.1 for device fix by @ctjlewis in https://github.com/huggingface/trl/pull/2766
📐 Add vLLM dtype configuration for GRPO trainer by @joey00072 in https://github.com/huggingface/trl/pull/2738
📖 Clarification max len in Reward documentation by @ParagEkbote in https://github.com/huggingface/trl/pull/2740
🔎 Add missing script argument in PPO documentation by @JohnConnor123 in https://github.com/huggingface/trl/pull/2720
🤖 Properly unwrap torch.compile-ed models in GRPO by @winglian in https://github.com/huggingface/trl/pull/2750
🔁 🦈 Support iterative GRPO by @shirinyamani in https://github.com/huggingface/trl/pull/2700
🚧 Add Optional ZeRO-3 Weight Gathering for GRPO in Sequence Generation by @SeungyounShin in https://github.com/huggingface/trl/pull/2667
↔️ GRPO: Set max_model_len when initializing vLLM instance by @mirceapricop in https://github.com/huggingface/trl/pull/2728
💡 GRPO vram-efficiency improvement; only compute relevant logprobs by @tyler-romero in https://github.com/huggingface/trl/pull/2773
🙃 Fix reward function in GRPO example by @junuMoon in https://github.com/huggingface/trl/pull/2777
💡 Add 'Post training an LLM for reasoning with GRPO in TRL' tutorial by @sergiopaniego in https://github.com/huggingface/trl/pull/2785
📉 Optimize GRPO memory usage by redefining per_device_batch_size as generations per device by @qgallouedec in https://github.com/huggingface/trl/pull/2776
🆚 Distinguish padding and eos when they differ by @binary-husky in https://github.com/huggingface/trl/pull/2793
🎯 [SFT] add token accuracy metric by @kashif in https://github.com/huggingface/trl/pull/2597
📠 Log completions for GRPO by @qgallouedec in https://github.com/huggingface/trl/pull/2772
🔬 SFT simplification by @qgallouedec in https://github.com/huggingface/trl/pull/2405
➖ Fix GRPO example in README by @qgallouedec in https://github.com/huggingface/trl/pull/2800
⛰️ Reduce peak vram consumption with efficient selective log_softmax by @tyler-romero in https://github.com/huggingface/trl/pull/2799
fix: typos in documentation files by @maximevtush in https://github.com/huggingface/trl/pull/2804
📤 GRPO refactor loading the model weights to vllm by @winglian in https://github.com/huggingface/trl/pull/2817
🫘 Add set_seed() call in GRPO to ensure unique seed for each process by @qgallouedec in https://github.com/huggingface/trl/pull/2824
⚖️ Add reward weight in multi-reward settings for GRPO by @hesamsheikh in https://github.com/huggingface/trl/pull/2676
🙌 Share vLLM device with training when only 1 available by @qgallouedec in https://github.com/huggingface/trl/pull/2827
👴 Update tokenizer parameter to processing_class in tests by @qgallouedec in https://github.com/huggingface/trl/pull/2828
🥾 Allow bootstrap GRPO by @qgallouedec in https://github.com/huggingface/trl/pull/2829
⚡ Fix GRPO PEFT by @qgallouedec in https://github.com/huggingface/trl/pull/2725
Fix PeftModel check when moving weights to vlllm by @edbeeching in https://github.com/huggingface/trl/pull/2850
🪆 Fix for Incorrect ValueError Handling in reward_weights in grpo_trainer.py by @loveychen in https://github.com/huggingface/trl/pull/2843
👨‍👩‍👧 GRPO + PEFT + vLLM by @winglian in https://github.com/huggingface/trl/pull/2818

New Contributors

@nnsW3 made their first contribution in https://github.com/huggingface/trl/pull/2682
@sdpkjc made their first contribution in https://github.com/huggingface/trl/pull/2672
@famouswizard made their first contribution in https://github.com/huggingface/trl/pull/2691
@brawncode made their first contribution in https://github.com/huggingface/trl/pull/2699
@ParagEkbote made their first contribution in https://github.com/huggingface/trl/pull/2722
@defiberrys made their first contribution in https://github.com/huggingface/trl/pull/2727
@ctjlewis made their first contribution in https://github.com/huggingface/trl/pull/2766
@joey00072 made their first contribution in https://github.com/huggingface/trl/pull/2738
@JohnConnor123 made their first contribution in https://github.com/huggingface/trl/pull/2720
@shirinyamani made their first contribution in https://github.com/huggingface/trl/pull/2700
@mirceapricop made their first contribution in https://github.com/huggingface/trl/pull/2728
@tyler-romero made their first contribution in https://github.com/huggingface/trl/pull/2773
@junuMoon made their first contribution in https://github.com/huggingface/trl/pull/2777
@binary-husky made their first contribution in https://github.com/huggingface/trl/pull/2793
@maximevtush made their first contribution in https://github.com/huggingface/trl/pull/2804
@hesamsheikh made their first contribution in https://github.com/huggingface/trl/pull/2676
@loveychen made their first contribution in https://github.com/huggingface/trl/pull/2843

Full Changelog: https://github.com/huggingface/trl/compare/v0.9.6...v0.15.0