pypi / accelerate
accelerate
What changed in accelerate 1, from its official release notes, upgrade guides and advisories. Sources checked 2026-10-24.
5 breaking1 deprecations6 new APIs and options3 performance7 behavior and defaults47 fixes and other
See which of these apply to your code, ranked, in your agent:
npx @unstale/cli initaccelerate 169 changes
Breaking changes 5
Refuses context parallelism for models with sliding-window or chunked attention layers.
FSDP now raises a clear error when enabled on a mesh without a shard dimension.
Leaves an already-DTensor-sharded model on its existing device instead of moving it.
Removes the 8bit force hook for bnb.
XPU device handling uses spawn instead of fork, removes the IPEX dependency, and improves device-agnostic code.
Deprecations 1
Adds deprecation warnings for MS-AMP.
New APIs and options 6
Adds an FSDP2-only option to move checkpointed layer inputs to pinned CPU memory, reducing activation memory while preserving plain activation-checkpointing gradients.
Supports dispatching an entire model to disk, including tied weights, for offloading large models.
Adds a Neuron backend for torch dynamo, enabling --torch-compile with the Transformers Trainer.
Registers custom trackers by name so they can be selected through Accelerator's log_with argument.
Offloading support is added for Torchao.
Adds support for AWS Neuron Trainium and Inferentia devices.
Performance 3
FSDP2 activation checkpointing now wraps the matched transformer layer itself rather than each child, so inter-child activations are not all kept for backward.
Padded allgather and broadcast reduce recompilation on Neuron devices.
Accelerate imports faster by deferring heavy dependencies.
Behavior and default changes 7
Accelerate now works end-to-end on AMD ROCm devices.
Supports dynamic batch sizes when even_batches is enabled.
FSDP2 now shards embeddings and norms.
The step parameter defaults to None.
Batches can contain non-Tensor values when dispatch_batches=True.
Changes the default value of fsdp_min_num_params to an integer.
Disables compile hooks lazily.
Fixes and other changes 47
Reports MPS as the accelerator and reads the MPS chip name.
Handles a missing accelerate executable in the env command.
Fixes the --cpu option so it sets the KMP variables.
Reaches through the DeepSpeed optimizer wrapper when calling eval().
Fixes compile_regions so it does not run the uncompiled module.
Fixes conversion so the first and last linear layers are not converted.
Preserves dataloader iteration state when resuming from a checkpoint.
Fixes a crash when disk offloading FP8 tensors.
Fixes the command for timm >= 1.0.29 by adding the hf-hub: prefix.
Treats MPS out-of-memory errors as OOM when finding an executable batch size.
Forwards reduce_batch_size_fn through the decorator form of find_executable_batch_size.
Fixes path resolution for FSDP sharded checkpoints.
Fixes tied-embedding models on torch >= 2.13 by placing the output embedding in the same fully_shard group.
Fixes FSDP2/PEFT checkpointing that dropped every rank's adapter shard except rank 0.
Stops the function from mutating module._non_persistent_buffers_set.
Gradient clipping now supports DTensor-sharded models when plain tensors and DTensors are mixed, avoiding a foreach-op failure.
Fixes memory accounting for integrated CUDA devices.
Restores all RNG backends rather than only one.
Fixes the setting so MLflow run nesting can be turned off.
Uses per-rank torch.save/load for FSDP2 sharded state dictionaries to fix checkpoint timeouts on large NPU setups.
Corrects byte-size handling for FP8 fnuz and e8m0fnu dtypes.
FSDP2 casts models to a uniform dtype before sharding and excludes non-floating frozen Params4bit parameters to prevent QLoRA crashes.
Fixes full-state-dict loading, including dtype mismatches and key-based matching, and adds a guard for Params4bit models.
The FSDP2 auto-wrap policy no longer ignores the _no_split_modules fallback.
Guards against None kwargs in tracker initialization and logging.
Fixes hook detachment statistics restoration for int8 offload.
Fixes the sharding condition when the number of shards equals the number of processes.
Fixes keep_in_fp32_modules handling for tied weights.
Prevents models from being wrapped twice.
Avoids implicit padding when apply_padding is false and num_samples is smaller than num_processes.
Drops REMOVED and NOT_YET_IMPLEMENTED FSDP1 keys instead of leaking them during conversion.
Passes the step parameter through instead of ignoring it.
Fixes async compatibility across Python versions.
Fixes the CPU offload check and a bug when cpu_ram_efficient_loading and cpu_offload are both enabled.
Improves DeepSpeed sequence-parallel integration, including evaluation during training and process-group handling; errors when context and sequence parallelism are both enabled.
Fixes a KeyError when extracting a model with partial torch.compile.
Fixes torchao default configuration with padding and FSDP2 all-gather support, and execution with Transformer Engine.
Fixes parameter upcasting, tied embedding errors, DCP optimizer-state loading, bfloat16 optimizer-step crashes, and ignored_params crashes with torch versions below 2.7.0.
Fixes setting and loading the HPU RNG state.
Fixes logging logic when in_order is set to True.
Fixes KMP_AFFINITY being incorrectly set for non-CPU training.
Fixes a mutable default in Megatron initialization and an IndexError on an empty ModuleList.
Avoids using nvidia-smi on CPU-only Colab instances.
Fixes an error when parallelism_config is None.
Fixes device-index comparison for hf_device_map.
Fixes a KeyError with num_workers greater than zero and fixes stateful dataloader behavior with DDP.
Avoids duplicate W&B initialization in offline mode.