v2.0.2

Tl;dr

• New models (idefics2, phi3)
• Cleaner VLM support in the openai layer
• Upgraded to pytorch 2.3.0

What's Changed

• Make --cuda-graphs 0 work as expected (bis) by @fxmarty in https://github.com/huggingface/text-generation-inference/pull/1768
• fix typos in docs and add small clarifications by @MoritzLaurer in https://github.com/huggingface/text-generation-inference/pull/1790
• Add attribute descriptions for GenerateParameters by @Wauplin in https://github.com/huggingface/text-generation-inference/pull/1798
• feat: allow null eos and bos tokens in config by @drbh in https://github.com/huggingface/text-generation-inference/pull/1791
• Phi3 support by @Narsil in https://github.com/huggingface/text-generation-inference/pull/1797
• Idefics2. by @Narsil in https://github.com/huggingface/text-generation-inference/pull/1756
• fix: avoid frequency and repetition penalty on padding tokens by @drbh in https://github.com/huggingface/text-generation-inference/pull/1765
• Adding support for HF_HUB_OFFLINE support in the router. by @Narsil in https://github.com/huggingface/text-generation-inference/pull/1789
• feat: improve temperature logic in chat by @drbh in https://github.com/huggingface/text-generation-inference/pull/1749
• Updating the benchmarks so everyone uses openai compat layer. by @Narsil in https://github.com/huggingface/text-generation-inference/pull/1800
• Update guidance docs to reflect grammar support in API by @dr3s in https://github.com/huggingface/text-generation-inference/pull/1775
• Use the generation config. by @Narsil in https://github.com/huggingface/text-generation-inference/pull/1808
• 2nd round of benchmark modifications (tiny adjustements to avoid overloading the host). by @Narsil in https://github.com/huggingface/text-generation-inference/pull/1816
• Adding new env variables for TPU backends. by @Narsil in https://github.com/huggingface/text-generation-inference/pull/1755
• add intel xpu support for TGI by @sywangyi in https://github.com/huggingface/text-generation-inference/pull/1475
• Blunder by @Narsil in https://github.com/huggingface/text-generation-inference/pull/1815
• Fixing qwen2. by @Narsil in https://github.com/huggingface/text-generation-inference/pull/1818
• Dummy CI run. by @Narsil in https://github.com/huggingface/text-generation-inference/pull/1817
• Changing the waiting_served_ratio default (stack more aggressively by default). by @Narsil in https://github.com/huggingface/text-generation-inference/pull/1820
• Better graceful shutdown. by @Narsil in https://github.com/huggingface/text-generation-inference/pull/1827
• Add the missing tool_prompt parameter to Python client by @maziyarpanahi in https://github.com/huggingface/text-generation-inference/pull/1825
• Small CI cleanup. by @Narsil in https://github.com/huggingface/text-generation-inference/pull/1801
• Add reference to TPU support by @brandonroyal in https://github.com/huggingface/text-generation-inference/pull/1760
• fix: use get_speculate to the number of layers by @OlivierDehaene in https://github.com/huggingface/text-generation-inference/pull/1737
• feat: add how it works section by @drbh in https://github.com/huggingface/text-generation-inference/pull/1773
• Fixing frequency penalty by @martinigoyanes in https://github.com/huggingface/text-generation-inference/pull/1811
• feat: add vlm docs and simple examples by @drbh in https://github.com/huggingface/text-generation-inference/pull/1812
• Handle images in chat api by @drbh in https://github.com/huggingface/text-generation-inference/pull/1828
• chore: update torch by @OlivierDehaene in https://github.com/huggingface/text-generation-inference/pull/1730
• (chore): torch 2.3.0 by @Narsil in https://github.com/huggingface/text-generation-inference/pull/1833

New Contributors

• @MoritzLaurer made their first contribution in https://github.com/huggingface/text-generation-inference/pull/1790
• @dr3s made their first contribution in https://github.com/huggingface/text-generation-inference/pull/1775
• @maziyarpanahi made their first contribution in https://github.com/huggingface/text-generation-inference/pull/1825
• @brandonroyal made their first contribution in https://github.com/huggingface/text-generation-inference/pull/1760
• @martinigoyanes made their first contribution in https://github.com/huggingface/text-generation-inference/pull/1811

Full Changelog: https://github.com/huggingface/text-generation-inference/compare/v2.0.1...v2.0.2