Text Embeddings Inference

What's Changed

• Use rust-toolchain.toml before rustup on Dockerfile-{cuda,cuda-all} by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/842
• fix(backend): replace bare except with Exception in device check by @llukito in…

What's Changed

• Fix auto-truncate false setting by @vrdn-23 in https://github.com/huggingface/text-embeddings-inference/pull/836
• Set pad_token_id as nullable & add support for rope_parameters by @alvarobartt in…

What's Changed

🚨 Fix

• Fix support for containers w/ CUDA 13.0+ by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/831

When releasing ghcr.io/huggingface/text-embeddings-inference:cuda-1.9 with CUDA 12.9 and cuda-compat-12-9 there…

What's changed?

🚨 Breaking changes

• Default HiddenAct::Gelu to GeLU + tanh in favour of GeLU erf by…

What's Changed

Bug Fixes

• Fix error code for empty requests by @vrdn-23 in https://github.com/huggingface/text-embeddings-inference/pull/727
• Fix the infinite loop when max_input_length is bigger than max-batch-tokens by @kozistr in…

🔧 Fixed Intel MKL Support

Since Text Embeddings Inference (TEI) v1.7.0, Intel MKL support had been broken due to changes in the candle dependency. Neither static-linking nor dynamic-linking worked correctly, which caused models using Intel MKL on CPU to fail with…

Today, Google releases EmbeddingGemma, a state-of-the-art multilingual embedding model perfect for…

Notable Changes

• Qwen3 support for 0.6B, 4B and 8B on CPU, MPS, and FlashQwen3 on CUDA and Intel HPUs -…

Noticeable Changes

Qwen3 was not working fine on CPU / MPS when sending batched requests on FP16 precision, due to the FP32 minimum value downcast (now manually set to FP16 minimum value instead) leading to null values, as well as a missing to_dtype call on the…

Noticeable Changes

Qwen3 support included for Intel HPU, and fixed for CPU / Metal / CUDA.

What's Changed

• Default to Qwen3 in README.md and docs/ examples by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/641
• Fix Qwen3 by…

Notable change

• Added support for Qwen3 embeddigns

What's Changed

• Adding suggestions to fixing missing ONNX files. by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/624
• Add Qwen3Model by @alvarobartt in…

What's Changed

• [Docs] Update quick tour by @NielsRogge in https://github.com/huggingface/text-embeddings-inference/pull/574
• Update README.md and supported_models.md by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/572
• Back with…

Notable changes

• Upgrade dependencies heavily (candle 0.5 -> 0.8 and related)
• Added ModernBert support by @kozistr !

What's Changed

• Moving cublaslt into TEI extension for easier upgrade of candle globally by @Narsil in…

What's Changed

• Enable intel devices CPU/XPU/HPU for python backend by @yuanwu2017 in https://github.com/huggingface/text-embeddings-inference/pull/245
• add reranker model support for python backend by @kaixuanliu in…

What's Changed

• feat: support multiple backends at the same time by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/440
• feat: GTE classification head by @kozistr in https://github.com/huggingface/text-embeddings-inference/pull/441 *…

What's Changed

• Download model.onnx_data by @kozistr in https://github.com/huggingface/text-embeddings-inference/pull/343
• Rename 'Sentence Transformers' to 'sentence-transformers' in docstrings by @Wauplin in…

Notable Changes

• ONNX runtime for CPU deployments: greatly improve CPU deployment throughput
• Add /similarity route

What's Changed

• tokenizer max limit on input size by @ErikKaum in https://github.com/huggingface/text-embeddings-inference/pull/324
• docs:…

Notable Changes

• Cuda support for the Qwen2 model architecture

What's Changed

• feat(candle): support Qwen2 on Cuda by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/316
• fix(candle): fix last token pooling

Full Changelog:…

Notable changes

• New truncation direction parameter
• Cuda support for JinaCode model architecture
• Cuda support for Mistral model architecture
• Cuda…

What's Changed

• fix limit peak memory to build cuda-all docker image by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/246

Full Changelog: https://github.com/huggingface/text-embeddings-inference/compare/v1.2.2...v1.2.3