What's New

Support for Intel IPEX support has been added, check out the how-to guide now!
Various modifications have been added to begin work on having 🤗 Accelerate be the foundation for the Trainer, keep an eye on the repos to see how our progress is coming along!
FP8 training is now supported on Ada Lovelance GPUs
The wandb integration now supports logging of images and tables through tracker.log_images and tracker.log_tables respectively
Many, many squashed bugs! (see the full detailed report for just what they were)
17 new contributors to the framework, congratulations to all who took their first step! 🚀

What's Changed

Fix pypi image by @muellerzr in https://github.com/huggingface/accelerate/pull/1249
raise error when dataloader with None as batch_size when using DS by @pacman100 in https://github.com/huggingface/accelerate/pull/1250
Handle missing deepspeed config by @HeyangQin in https://github.com/huggingface/accelerate/pull/1251
[core] Add Quantization support for dispatch_model by @younesbelkada in https://github.com/huggingface/accelerate/pull/1237
Check attribute 'overflow' exists in optimizer. by @tensimiku in https://github.com/huggingface/accelerate/pull/1259
ipex intel extension for pytorch integration by @sywangyi in https://github.com/huggingface/accelerate/pull/1255
fix issue template by @stas00 in https://github.com/huggingface/accelerate/pull/1264
Change error raised to ValueError by @sgugger in https://github.com/huggingface/accelerate/pull/1267
Fix reduce operation by @xyfJASON in https://github.com/huggingface/accelerate/pull/1268
Raise import error if fp8 not available in has_transfomer_engine_layers by @muellerzr in https://github.com/huggingface/accelerate/pull/1283
Add missing FP8 options to CLI by @muellerzr in https://github.com/huggingface/accelerate/pull/1284
Update quicktour.mdx by @standardAI in https://github.com/huggingface/accelerate/pull/1273
Minor fix whitespace colon by @guspan-tanadi in https://github.com/huggingface/accelerate/pull/1272
fix attribute error in DataloaderShared by @ZhiyuanChen in https://github.com/huggingface/accelerate/pull/1278
Fix TypeError bug in honor_type by @muellerzr in https://github.com/huggingface/accelerate/pull/1285
Raise more explicit error when transformer_engine isn't installed by @muellerzr in https://github.com/huggingface/accelerate/pull/1287
Expound error on recursively_apply by @muellerzr in https://github.com/huggingface/accelerate/pull/1286
Only check for dtype if it has it in get_state_dict by @muellerzr in https://github.com/huggingface/accelerate/pull/1288
[bnb] fix bnb slow test by @younesbelkada in https://github.com/huggingface/accelerate/pull/1292
Raise better error on notebook_launcher by @muellerzr in https://github.com/huggingface/accelerate/pull/1293
Make note about grad accum and prec in performance documentation by @muellerzr in https://github.com/huggingface/accelerate/pull/1296
fix for load_checkpoint_and_dispatch(device_map=None) by @anentropic in https://github.com/huggingface/accelerate/pull/1297
Set the state device dependant to Accelerator on multigpu by @muellerzr in https://github.com/huggingface/accelerate/pull/1220
add usage guide for ipex plugin by @sywangyi in https://github.com/huggingface/accelerate/pull/1270
Simplify MPS implementation by @sgugger in https://github.com/huggingface/accelerate/pull/1308
Bug fix in setattr by @aashiqmuhamed in https://github.com/huggingface/accelerate/pull/1312
Allow xpu backend by @muellerzr in https://github.com/huggingface/accelerate/pull/1313
Default to nccl by @muellerzr in https://github.com/huggingface/accelerate/pull/1314
offload the previous module hook before the current module is moved to… by @williamberman in https://github.com/huggingface/accelerate/pull/1315
Ensure that dynamo is compatible with mixed precision by @muellerzr in https://github.com/huggingface/accelerate/pull/1318
Upgrade torch version on main tests by @muellerzr in https://github.com/huggingface/accelerate/pull/1323
Add test flag and import check for dynamo by @muellerzr in https://github.com/huggingface/accelerate/pull/1322
ensure module prefixes only match that module by @xloem in https://github.com/huggingface/accelerate/pull/1319
remove repetitive entries from device lists by @xloem in https://github.com/huggingface/accelerate/pull/1321
Fix failing test on main by @muellerzr in https://github.com/huggingface/accelerate/pull/1332
Verbosity, Progress Bar for Loading by @xloem in https://github.com/huggingface/accelerate/pull/1329
Skip failing torch 2.0+ test by @muellerzr in https://github.com/huggingface/accelerate/pull/1339
Remove unused amp import util by @muellerzr in https://github.com/huggingface/accelerate/pull/1340
Fix nested context manager for main_process_first() by @flukeskywalker in https://github.com/huggingface/accelerate/pull/1304
Small progress bar fix by @xloem in https://github.com/huggingface/accelerate/pull/1341
Pop more backend options by @muellerzr in https://github.com/huggingface/accelerate/pull/1342
Support FP8 mixed precision training for Ada Lovelace GPUs by @Dango233 in https://github.com/huggingface/accelerate/pull/1348
using deepspeed.comm for distrbiuted init by @pacman100 in https://github.com/huggingface/accelerate/pull/1352
[bnb] Fix bnb slow test by @younesbelkada in https://github.com/huggingface/accelerate/pull/1355
Better check for packages availability by @apbard in https://github.com/huggingface/accelerate/pull/1356
fix: typing issues, and replace deprecated python typing (Optional, Union) to | by @kiyoon in https://github.com/huggingface/accelerate/pull/1363
Fix default FSDP_MIN_NUM_PARAMS so it's an int by @sam-hieken in https://github.com/huggingface/accelerate/pull/1367
Special transformers case from args by @muellerzr in https://github.com/huggingface/accelerate/pull/1364
Improve accelerate env reporting by @muellerzr in https://github.com/huggingface/accelerate/pull/1376
Seperate out contextmanager generation by @muellerzr in https://github.com/huggingface/accelerate/pull/1379
delete textfile after tests are done by @muellerzr in https://github.com/huggingface/accelerate/pull/1381
Fix flakey thread issue by @muellerzr in https://github.com/huggingface/accelerate/pull/1387
fix config bug for 'mixed_precision' from 'yaml.safe_load()' by @ys-eric-choi in https://github.com/huggingface/accelerate/pull/1386
Log Images and other types to wandb by @tcapelle in https://github.com/huggingface/accelerate/pull/962
Bump torch version by @muellerzr in https://github.com/huggingface/accelerate/pull/1392
Fix gather_obj by @muellerzr in https://github.com/huggingface/accelerate/pull/1391
Update training_zoo.mdx by @yuvalkirstain in https://github.com/huggingface/accelerate/pull/1397

New Contributors

@HeyangQin made their first contribution in https://github.com/huggingface/accelerate/pull/1251
@tensimiku made their first contribution in https://github.com/huggingface/accelerate/pull/1259
@xyfJASON made their first contribution in https://github.com/huggingface/accelerate/pull/1268
@standardAI made their first contribution in https://github.com/huggingface/accelerate/pull/1273
@guspan-tanadi made their first contribution in https://github.com/huggingface/accelerate/pull/1272
@anentropic made their first contribution in https://github.com/huggingface/accelerate/pull/1297
@aashiqmuhamed made their first contribution in https://github.com/huggingface/accelerate/pull/1312
@williamberman made their first contribution in https://github.com/huggingface/accelerate/pull/1315
@xloem made their first contribution in https://github.com/huggingface/accelerate/pull/1319
@flukeskywalker made their first contribution in https://github.com/huggingface/accelerate/pull/1304
@Dango233 made their first contribution in https://github.com/huggingface/accelerate/pull/1348
@apbard made their first contribution in https://github.com/huggingface/accelerate/pull/1356
@kiyoon made their first contribution in https://github.com/huggingface/accelerate/pull/1363
@sam-hieken made their first contribution in https://github.com/huggingface/accelerate/pull/1367
@ys-eric-choi made their first contribution in https://github.com/huggingface/accelerate/pull/1386
@tcapelle made their first contribution in https://github.com/huggingface/accelerate/pull/962
@yuvalkirstain made their first contribution in https://github.com/huggingface/accelerate/pull/1397

Full Changelog: https://github.com/huggingface/accelerate/compare/v0.18.0...v0.19.0