v1.6.0: FSDPv2, DeepSpeed TP and XCCL backend support
This release introduces the support for FSDPv2 thanks to @S1ro1.
If you are using python code, you need to set fsdp_version=2 in FullyShardedDataParallelPlugin:
from accelerate import FullyShardedDataParallelPlugin, Accelerator
fsdp_plugin = FullyShardedDataParallelPlugin(
fsdp_version=2
# other options...
)
accelerator = Accelerator(fsdp_plugin=fsdp_plugin)
If want to convert a YAML config that contains the FSDPv1 config to FSDPv2 one , use our conversion tool:
accelerate to-fsdp2 --config_file config.yaml --output_file new_config.yaml`
To learn more about the difference between FSDPv1 and FSDPv2, read the following documentation.
We have added initial support for DeepSpeed + TP. Not many changes were required as the DeepSpeed APIs was already compatible. We only needed to make sure that the dataloader was compatible with TP and that we were able to save the TP weights. Thanks @inkcherry for the work ! https://github.com/huggingface/accelerate/pull/3390.
To use TP with deepspeed, you need to update the setting in the deepspeed config file by including tensor_parallel key:
....
"tensor_parallel":{
"autotp_size": ${autotp_size}
},
...
More details in this deepspeed PR.
We've added support for XCCL which is an Intel distributed backend which can be used with XPU devices. More details in this torch PR. Thanks @dvrogozh for the integration !
log_artifact, log_artifacts and log_figure capabilities to the MLflowTracker. by @luiz0992 in https://github.com/huggingface/accelerate/pull/3419Full Changelog: https://github.com/huggingface/accelerate/compare/v1.5.2...v1.6.0
Fetched April 7, 2026