v0.4.0: peft integration

Apply RLHF and fine-tune your favorite large model on consumer GPU using peft and trl ! Share also easily your trained RLHF adapters on the Hub with few lines of code

With this integration you can train gpt-neo-x (20B parameter model - 40GB in bfloat16) on a 24GB consumer GPU!

What's Changed

• Allow running evaluate-toxicity with cpu by @jordimas in https://github.com/lvwerra/trl/pull/195
• [core] Fix quality issue by @younesbelkada in https://github.com/lvwerra/trl/pull/197
• Add 1.12.1 torch compatibility in sum method by @PanchenkoYehor in https://github.com/lvwerra/trl/pull/190
• peft integration by @edbeeching in https://github.com/lvwerra/trl/pull/163
• [core] Update dependency by @younesbelkada in https://github.com/lvwerra/trl/pull/206

New Contributors

• @PanchenkoYehor made their first contribution in https://github.com/lvwerra/trl/pull/190

Full Changelog: https://github.com/lvwerra/trl/compare/v0.3.1...v0.4.0