v0.6.0 — Accelerate — releases.sh

This release adds support for bloat16 mixed precision training (requires PyTorch >= 1.10) and a brand-new checkpoint utility to help with resuming interrupted trainings. We also get a completely revamped documentation frontend.

Checkpoints

Save the current state of all your objects (models, optimizers, RNG states) with accelerator.save_state(path_to_checkpoint) and reload everything by calling accelerator.load_state(path_to_checkpoint)

Add in checkpointing capability by @muellerzr in https://github.com/huggingface/accelerate/pull/255
Implementation of saving and loading custom states by @muellerzr in https://github.com/huggingface/accelerate/pull/270

BFloat16 support

Accelerate now supports bfloat16 mixed precision training. As a result the old --fp16 argument has been deprecated to be replaced by the more generic --mixed-precision.

Add bfloat16 support #243 by @ikergarcia1996 in https://github.com/huggingface/accelerate/pull/247

New env subcommand

You can now type accelerate env to have a copy-pastable summary of your environment and default configuration. Very convenient when opening a new issue!

add env command by @johnnv1 in https://github.com/huggingface/accelerate/pull/280

New doc frontend

The documentation has been switched to the new Hugging Face frontend, like Transformers and Datasets.

Convert documentation to the new front by @sgugger in https://github.com/huggingface/accelerate/pull/271

What's Changed

Fix send_to_device with non-tensor data by @sgugger in https://github.com/huggingface/accelerate/pull/177
Handle UserDict in all utils by @sgugger in https://github.com/huggingface/accelerate/pull/179
Use collections.abc.Mapping to handle both the dict and the UserDict types by @mariosasko in https://github.com/huggingface/accelerate/pull/180
fix: use store_true on argparse in nlp example by @monologg in https://github.com/huggingface/accelerate/pull/183
Update README.md by @TevenLeScao in https://github.com/huggingface/accelerate/pull/187
Add signature check for set_to_none in Optimizer.zero_grad by @sgugger in https://github.com/huggingface/accelerate/pull/189
fix typo in code snippet by @MrZilinXiao in https://github.com/huggingface/accelerate/pull/199
Add high-level API reference to README by @Chris-hughes10 in https://github.com/huggingface/accelerate/pull/204
fix rng_types in accelerator by @s-kumano in https://github.com/huggingface/accelerate/pull/206
Pass along drop_last in DispatchDataLoader by @sgugger in https://github.com/huggingface/accelerate/pull/212
Rename state to avoid name conflicts with pytorch's Optimizer class. by @yuxinyuan in https://github.com/huggingface/accelerate/pull/224
Fix lr scheduler num samples by @sgugger in https://github.com/huggingface/accelerate/pull/227
Add customization point for init_process_group kwargs by @sgugger in https://github.com/huggingface/accelerate/pull/228
Fix typo in installation docs by @jaketae in https://github.com/huggingface/accelerate/pull/234
make deepspeed optimizer match parameters of passed optimizer by @jmhessel in https://github.com/huggingface/accelerate/pull/246
Upgrade black to version ~=22.0 by @LysandreJik in https://github.com/huggingface/accelerate/pull/250
add support of gather_object by @ZhiyuanChen in https://github.com/huggingface/accelerate/pull/238
Add launch flags --module and --no_python (#256) by @parameter-concern in https://github.com/huggingface/accelerate/pull/258
Accelerate + Animus/Catalyst = 🚀 by @Scitator in https://github.com/huggingface/accelerate/pull/249
Add debug_launcher by @sgugger in https://github.com/huggingface/accelerate/pull/259
enhance compatibility of honor type by @ZhiyuanChen in https://github.com/huggingface/accelerate/pull/241
Add a flag to use CPU only in the config by @sgugger in https://github.com/huggingface/accelerate/pull/263
Basic fixes for DeepSpeed by @sgugger in https://github.com/huggingface/accelerate/pull/264
Ability to set the seed with randomness from inside Accelerate by @muellerzr in https://github.com/huggingface/accelerate/pull/266
Don't use dispatch_batches when torch is < 1.8.0 by @sgugger in https://github.com/huggingface/accelerate/pull/269
Make accelerated model with AMP possible to pickle by @BenjaminBossan in https://github.com/huggingface/accelerate/pull/274
Contributing guide by @LysandreJik in https://github.com/huggingface/accelerate/pull/254
replace texts and link (master -> main) by @johnnv1 in https://github.com/huggingface/accelerate/pull/282
Use workflow from doc-builder by @sgugger in https://github.com/huggingface/accelerate/pull/275
Pass along execution info to the exit of autocast by @sgugger in https://github.com/huggingface/accelerate/pull/284

New Contributors

@mariosasko made their first contribution in https://github.com/huggingface/accelerate/pull/180
@monologg made their first contribution in https://github.com/huggingface/accelerate/pull/183
@TevenLeScao made their first contribution in https://github.com/huggingface/accelerate/pull/187
@MrZilinXiao made their first contribution in https://github.com/huggingface/accelerate/pull/199
@Chris-hughes10 made their first contribution in https://github.com/huggingface/accelerate/pull/204
@s-kumano made their first contribution in https://github.com/huggingface/accelerate/pull/206
@yuxinyuan made their first contribution in https://github.com/huggingface/accelerate/pull/224
@jaketae made their first contribution in https://github.com/huggingface/accelerate/pull/234
@jmhessel made their first contribution in https://github.com/huggingface/accelerate/pull/246
@ikergarcia1996 made their first contribution in https://github.com/huggingface/accelerate/pull/247
@ZhiyuanChen made their first contribution in https://github.com/huggingface/accelerate/pull/238
@parameter-concern made their first contribution in https://github.com/huggingface/accelerate/pull/258
@Scitator made their first contribution in https://github.com/huggingface/accelerate/pull/249
@muellerzr made their first contribution in https://github.com/huggingface/accelerate/pull/255
@BenjaminBossan made their first contribution in https://github.com/huggingface/accelerate/pull/274
@johnnv1 made their first contribution in https://github.com/huggingface/accelerate/pull/280

Full Changelog: https://github.com/huggingface/accelerate/compare/v0.5.1...v0.6.0