pypi / peft
peft
What changed in peft 0.21, from its official release notes, upgrade guides and advisories. Sources checked 2026-10-24.
6 breaking2 new APIs and options2 performance2 behavior and defaults6 fixes and other
See which of these apply to your code, ranked, in your agent:
npx @unstale/cli initpeft 0.2118 changes
Breaking changes 6
PEFT requires Transformers ≥ 5.17.0 to work with Tensor Parallel.
Hot-swapping adapters is refused while they are merged into the base weights.
Adding or loading a second adaption prompt adapter no longer automatically activates it, matching the behavior of other PEFT methods.
Adding a second adapter that targets non-existing layers now raises an error instead of silently creating an adapter that adapts nothing. Catch peft.PeftError to suppress potential errors.
Deleting an adapter that is merged now raises an error because deletion would not remove its effect.
Direct injection now rejects PEFT methods with shared state that would live on a parent model, such as VeRA, because no such parent model exists.
New APIs and options 2
Prompt learning methods now allow deleting an adapter.
A new LoRA-specific optimizer applies Riemannian pre-conditioning to gradients to improve feature learning and avoid separately tuning learning rates for LoRA's A and B matrices.
Performance 2
For the common case of one active adapter, weight and MoE LoRA delta computation is folded into a single baddbmm operation for speed, with slight numerical differences expected.
A delta-based forward pass reduces memory footprint by 22% and training time by 46% on the MetaMathQA benchmark while keeping task accuracy and forgetting stable; slight numerical differences are expected, especially in fp16 and bf16.
Behavior and default changes 2
OFT moved from a method-specific quantization backend to the generic backend, affecting bitsandbytes, GPT-Q, and other quantization integrations; existing checkpoints should be unaffected.
Support was added for torchao v0.18.0 and newer tensor subclasses, including Int8Tensor and Float8Tensor.
Fixes and other changes 6
Fixes an issue that prevented encoder-decoder models from working with Transformers ≥ 5.18.0.
Tensor Parallel now works properly with PEFT, enabling developers to use PEFT with Tensor Parallel.
Loading or adding another adapter no longer changes requires_grad for existing adapters or the base model based on is_trainable.
Combining LoRA adapters no longer applies LoRA scaling twice in the SVD path.
Targeted module names no longer contain duplicates and remain sorted by insertion order.
Targeted parameter names no longer contain duplicates and remain sorted by insertion order.