v0.21.2
Fixes an issue that prevented encoder-decoder models from working when using Transformers 5.18.0 or later.
PEFT
$
npx @buildinternet/releases get peftMon
Wed
Fri
OctNovDecJanFebMarAprMayJunJulAugSepOct
LessMore
Releases4Avg1/moVersionsv0.20.0 to v0.21.2
v0.21.1
PEFT now works properly with Tensor Parallel (TP), and requires Transformers 5.17.0 or later.
v0.21.0
Four new fine-tuning methods ship: a Riemannian-preconditioned LoRA optimizer, the KaSA LoRA variant, Super-Tuning sparsity allocation, and ShadowPEFT. Also fixed a LoRA scaling applied twice in the SVD path of add_weighted_adapter, sped up MoE target parameter computation, and reduced Orthogonal Subspace Fine-tuning memory 22% and training time 46%.
v0.20.0
PEFT v0.20.0 adds nine new methods including HiRA, GLoRA, BEFT, MonteCLoRA, VeLoRA, Uni-LoRA, FRoD, MiCA, and DEFT, along with automatic LoRA target selection, broader quantization support for BOFT, MiSS, VeRA, and SHiRA, and a new image generation benchmark. AutoGPTQ is formally deprecated following the switch to GPT-QModel.
A small patch release containing these fixes:
- #3161
- #3165
Full Changelog: https://github.com/huggingface/peft/compare/v0.19.0...v0.19.1
Highlights
This PEFT release contains no less than nine new PEFT methods, described below. It also contains numerous enhancements that should make PEFT more useful to many users.
<img width="1248" height="560" alt="peft-v0 19 0"…
Small patch release containing the following changes:
- #2934: Small fixes required for some special cases to work with the upcoming transformers v5 release
- #2963: Fix to enable PEFT to run with AMD ROCm thanks to @vladmandic
- #2976: Fix a regression that inadvertently…
v0.18.0
Highlights
FIXME update list of all changes, so some more commits were added
New Methods
RoAd
@ppetrushkov added…
This patch release contains a few fixes (via #2710) for the newly introduced target_parameters feature, which allows LoRA to target nn.Parameters directly (useful for mixture of…
v0.17.0
v0.16.0
This patch fixes a bug that resulted in prompt learning methods like P-tuning not to work (#2477).
This patch includes a fix for #2450. In this bug modules_to_save was not handled correctly when used in conjunction with DeepSpeed ZeRO stage 3 which resulted in those modules being placeholder values in the saved checkpoints.
Full Changelog:…
v0.14.0
Highlights
New Methods
Context-aware Prompt Tuning
@tsachiblau added a new soft prompt method called [Context-aware Prompt Tuning…
v0.13.2
This patch release contains a small bug fix for an issue that prevented some LoRA checkpoints to be loaded correctly (mostly concerning stable diffusion checkpoints not trained with PEFT when loaded in diffusers, #2144).
Full Changelog:…
v0.13.1
This patch release contains a small bug fix for the low_cpu_mem_usage=True option (#2113).
Full Changelog: https://github.com/huggingface/peft/compare/v0.13.0...v0.13.1
v0.13.0
Highlights
New methods
LoRA+
@kallewoof added LoRA+ to PEFT (#1915). This is a function that allows to [initialize an…
v0.12.0
Highlights
New methods
OLoRA
@tokenizer-decode added support for a new LoRA initialization strategy called OLoRA (#1828).…
Patch release v0.11.1
Fix a bug that could lead to C++ compilation errors after importing PEFT (#1738 #1739).
Full Changelog: https://github.com/huggingface/peft/compare/v0.11.0...v0.11.1
