Inference

What's Changed

• Use rust-toolchain.toml before rustup on Dockerfile-{cuda,cuda-all} by @alvarobartt in https://github.com/huggingface/text-...

What's Changed

• Fix auto-truncate false setting by @vrdn-23 in https://github.com/huggingface/text-embeddings-inference/pull/836
• Set `pad_to...

What's Changed

• misc(gha): expose action cache url and runtime as secrets by @mfuntowicz in https://github.com/huggingface/text-generation-infere...

What's Changed

Bug Fixes

• Fix error code for empty requests by @vrdn-23 in https://github.com/huggingface/text-embeddings-inference/pull...

What's Changed

• Add missing backslash by @philsupertramp in https://github.com/huggingface/text-generation-inference/pull/3311
• Revert "feat: b...

🔧 Fixed Intel MKL Support

Since Text Embeddings Inference (TEI) v1.7.0, Intel MKL support had been broken due to changes in the candle depend...

<img width="1200" height="648" alt="text-embeddings-inference-v1 8 1-embedding-gemma(1)" src="https://github.com/user-attachments/assets/8ad8fb64-cee4...

What's Changed

• [gaudi] Refine rope memory, do not need to keep sin/cos cache per layer by @sywangyi in https://github.com/huggingface/text-gener...

<img width="3600" height="1944" alt="text-embeddings-inference-v1 8 0(2)" src="https://github.com/user-attachments/assets/50df05b6-3821-4e2a-8de0-3e5c...

Noticeable Changes

Qwen3 was not working fine on CPU / MPS when sending batched requests on FP16 precision, due to the FP32 minimum value downca...

Noticeable Changes

Qwen3 support included for Intel HPU, and fixed for CPU / Metal / CUDA.

What's Changed

• Default to Qwen3 in `README...

Fix for Neuron models exported with batch_size 1.

What's Changed

• [gaudi] gemma3 text and vlm model intial support. need to add sliding window...

Neuron backend update.

What's Changed

• Remove useless packages by @yuanwu2017 in https://github.com/huggingface/text-generation-inference/pull...

Notable change

• Added support for Qwen3 embeddigns

What's Changed

• Adding suggestions to fixing missing ONNX files. by @Narsil in https...

What's Changed

• [Docs] Update quick tour by @NielsRogge in https://github.com/huggingface/text-embeddings-inference/pull/574
• Update `README.md...

Gaudi improvements.

What's Changed

• upgrade to new vllm extension ops(fix issue in exponential bucketing) by @sywangyi in https://github.com/h...

This release updates TGI to Torch 2.7 and CUDA 12.8.

What's Changed

• change HPU warmup logic: seq length should be with exponential growth by ...

Notable changes

• Prefill chunking for VLMs.

What's Changed

• Fixing Qwen 2.5 VL (32B). by @Narsil in https://github.com/huggingface/text...