v1.1.0 — Accelerate — releases.sh

Internals:

Allow for a data_seed argument in https://github.com/huggingface/accelerate/pull/3150
Trigger weights_only=True by default for all compatible objects when checkpointing and saving with torch.save in https://github.com/huggingface/accelerate/pull/3036
Handle negative values for dim input in pad_across_processes in https://github.com/huggingface/accelerate/pull/3114
Enable cpu bnb distributed lora finetune in https://github.com/huggingface/accelerate/pull/3159

DeepSpeed

Support torch dynamo for deepspeed>=0.14.4 in https://github.com/huggingface/accelerate/pull/3069

Megatron

update Megatron-LM plugin code to version 0.8.0 or higher in https://github.com/huggingface/accelerate/pull/3174

Big Model Inference

New has_offloaded_params utility added in https://github.com/huggingface/accelerate/pull/3188

Examples

Florence2 distributed inference example in https://github.com/huggingface/accelerate/pull/3123

Full Changelog

Handle negative values for dim input in pad_across_processes by @mariusarvinte in https://github.com/huggingface/accelerate/pull/3114
Fixup DS issue with weakref by @muellerzr in https://github.com/huggingface/accelerate/pull/3143
Refactor scaler to util by @muellerzr in https://github.com/huggingface/accelerate/pull/3142
DS fix, continued by @muellerzr in https://github.com/huggingface/accelerate/pull/3145
Florence2 distributed inference example by @hlky in https://github.com/huggingface/accelerate/pull/3123
POC: Allow for a data_seed by @muellerzr in https://github.com/huggingface/accelerate/pull/3150
Adding multi gpu speech generation by @dame-cell in https://github.com/huggingface/accelerate/pull/3149
support torch dynamo for deepspeed>=0.14.4 by @oraluben in https://github.com/huggingface/accelerate/pull/3069
Fixup Zero3 + save_model by @muellerzr in https://github.com/huggingface/accelerate/pull/3146
Trigger weights_only=True by default for all compatible objects by @muellerzr in https://github.com/huggingface/accelerate/pull/3036
Remove broken dynamo test by @oraluben in https://github.com/huggingface/accelerate/pull/3155
fix version check bug in get_xpu_available_memory by @faaany in https://github.com/huggingface/accelerate/pull/3165
enable cpu bnb distributed lora finetune by @jiqing-feng in https://github.com/huggingface/accelerate/pull/3159
[Utils] has_offloaded_params by @kylesayrs in https://github.com/huggingface/accelerate/pull/3188
fix bnb by @eljandoubi in https://github.com/huggingface/accelerate/pull/3186
[docs] update neptune API by @faaany in https://github.com/huggingface/accelerate/pull/3181
docs: fix a wrong word in comment in src/accelerate/accelerate.py:1255 by @Rebornix-zero in https://github.com/huggingface/accelerate/pull/3183
[docs] use nn.module instead of tensor as model by @faaany in https://github.com/huggingface/accelerate/pull/3157
Fix typo by @kylesayrs in https://github.com/huggingface/accelerate/pull/3191
MLU devices : Checks if mlu is available via an cndev-based check which won't trigger the drivers and leave mlu by @huismiling in https://github.com/huggingface/accelerate/pull/3187
update Megatron-LM plugin code to version 0.8.0 or higher. by @eljandoubi in https://github.com/huggingface/accelerate/pull/3174
🚨 🚨 🚨 Goodbye Python 3.8! 🚨 🚨 🚨 by @muellerzr in https://github.com/huggingface/accelerate/pull/3194
Update transformers.deepspeed references from transformers 4.46.0 release by @loadams in https://github.com/huggingface/accelerate/pull/3196
eliminate dead code by @statelesshz in https://github.com/huggingface/accelerate/pull/3198
take torch.nn.Module model into account when moving to device by @faaany in https://github.com/huggingface/accelerate/pull/3167
[docs] add xpu part and fix bug in torchrun by @faaany in https://github.com/huggingface/accelerate/pull/3166
Models With Tied Weights Need Re-Tieing After FSDP Param Init by @fabianlim in https://github.com/huggingface/accelerate/pull/3154
add the missing xpu for local sgd by @faaany in https://github.com/huggingface/accelerate/pull/3163
typo fix in big_modeling.py by @a-r-r-o-w in https://github.com/huggingface/accelerate/pull/3207
[Utils] align_module_device by @kylesayrs in https://github.com/huggingface/accelerate/pull/3204

New Contributors

@mariusarvinte made their first contribution in https://github.com/huggingface/accelerate/pull/3114
@hlky made their first contribution in https://github.com/huggingface/accelerate/pull/3123
@dame-cell made their first contribution in https://github.com/huggingface/accelerate/pull/3149
@kylesayrs made their first contribution in https://github.com/huggingface/accelerate/pull/3188
@eljandoubi made their first contribution in https://github.com/huggingface/accelerate/pull/3186
@Rebornix-zero made their first contribution in https://github.com/huggingface/accelerate/pull/3183
@loadams made their first contribution in https://github.com/huggingface/accelerate/pull/3196

Full Changelog: https://github.com/huggingface/accelerate/compare/v1.0.1...v1.1.0