v1.1.0: Python 3.9 minimum, torch dynamo deepspeed support, and bug fixes

Internals:

• Allow for a data_seed argument in https://github.com/huggingface/accelerate/pull/3150
• Trigger weights_only=True by default for all compatible objects when checkpointing and saving with torch.save in https://github.com/huggingface/accelerate/pull/3036
• Handle negative values for dim input in pad_across_processes in https://github.com/huggingface/accelerate/pull/3114
• Enable cpu bnb distributed lora finetune in https://github.com/huggingface/accelerate/pull/3159

DeepSpeed

• Support torch dynamo for deepspeed>=0.14.4 in https://github.com/huggingface/accelerate/pull/3069

Megatron

• update Megatron-LM plugin code to version 0.8.0 or higher in https://github.com/huggingface/accelerate/pull/3174

Big Model Inference

• New has_offloaded_params utility added in https://github.com/huggingface/accelerate/pull/3188

Examples

• Florence2 distributed inference example in https://github.com/huggingface/accelerate/pull/3123

Full Changelog

• Handle negative values for dim input in pad_across_processes by @mariusarvinte in https://github.com/huggingface/accelerate/pull/3114
• Fixup DS issue with weakref by @muellerzr in https://github.com/huggingface/accelerate/pull/3143
• Refactor scaler to util by @muellerzr in https://github.com/huggingface/accelerate/pull/3142
• DS fix, continued by @muellerzr in https://github.com/huggingface/accelerate/pull/3145
• Florence2 distributed inference example by @hlky in https://github.com/huggingface/accelerate/pull/3123
• POC: Allow for a data_seed by @muellerzr in https://github.com/huggingface/accelerate/pull/3150
• Adding multi gpu speech generation by @dame-cell in https://github.com/huggingface/accelerate/pull/3149
• support torch dynamo for deepspeed>=0.14.4 by @oraluben in https://github.com/huggingface/accelerate/pull/3069
• Fixup Zero3 + save_model by @muellerzr in https://github.com/huggingface/accelerate/pull/3146
• Trigger weights_only=True by default for all compatible objects by @muellerzr in https://github.com/huggingface/accelerate/pull/3036
• Remove broken dynamo test by @oraluben in https://github.com/huggingface/accelerate/pull/3155
• fix version check bug in get_xpu_available_memory by @faaany in https://github.com/huggingface/accelerate/pull/3165
• enable cpu bnb distributed lora finetune by @jiqing-feng in https://github.com/huggingface/accelerate/pull/3159
• [Utils] has_offloaded_params by @kylesayrs in https://github.com/huggingface/accelerate/pull/3188
• fix bnb by @eljandoubi in https://github.com/huggingface/accelerate/pull/3186
• [docs] update neptune API by @faaany in https://github.com/huggingface/accelerate/pull/3181
• docs: fix a wrong word in comment in src/accelerate/accelerate.py:1255 by @Rebornix-zero in https://github.com/huggingface/accelerate/pull/3183
• [docs] use nn.module instead of tensor as model by @faaany in https://github.com/huggingface/accelerate/pull/3157
• Fix typo by @kylesayrs in https://github.com/huggingface/accelerate/pull/3191
• MLU devices : Checks if mlu is available via an cndev-based check which won't trigger the drivers and leave mlu by @huismiling in https://github.com/huggingface/accelerate/pull/3187
• update Megatron-LM plugin code to version 0.8.0 or higher. by @eljandoubi in https://github.com/huggingface/accelerate/pull/3174
• 🚨 🚨 🚨 Goodbye Python 3.8! 🚨 🚨 🚨 by @muellerzr in https://github.com/huggingface/accelerate/pull/3194
• Update transformers.deepspeed references from transformers 4.46.0 release by @loadams in https://github.com/huggingface/accelerate/pull/3196
• eliminate dead code by @statelesshz in https://github.com/huggingface/accelerate/pull/3198
• take torch.nn.Module model into account when moving to device by @faaany in https://github.com/huggingface/accelerate/pull/3167
• [docs] add xpu part and fix bug in torchrun by @faaany in https://github.com/huggingface/accelerate/pull/3166
• Models With Tied Weights Need Re-Tieing After FSDP Param Init by @fabianlim in https://github.com/huggingface/accelerate/pull/3154
• add the missing xpu for local sgd by @faaany in https://github.com/huggingface/accelerate/pull/3163
• typo fix in big_modeling.py by @a-r-r-o-w in https://github.com/huggingface/accelerate/pull/3207
• [Utils] align_module_device by @kylesayrs in https://github.com/huggingface/accelerate/pull/3204

New Contributors

• @mariusarvinte made their first contribution in https://github.com/huggingface/accelerate/pull/3114
• @hlky made their first contribution in https://github.com/huggingface/accelerate/pull/3123
• @dame-cell made their first contribution in https://github.com/huggingface/accelerate/pull/3149
• @kylesayrs made their first contribution in https://github.com/huggingface/accelerate/pull/3188
• @eljandoubi made their first contribution in https://github.com/huggingface/accelerate/pull/3186
• @Rebornix-zero made their first contribution in https://github.com/huggingface/accelerate/pull/3183
• @loadams made their first contribution in https://github.com/huggingface/accelerate/pull/3196

Full Changelog: https://github.com/huggingface/accelerate/compare/v1.0.1...v1.1.0