What's Changed

• fix(gke): accept null values for vertex env vars by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/243
• fix: fix cpu image to not default on the sagemaker entrypoint

Full Changelog: https://github.com/huggingface/text-embeddings-inference/compare/v1.2.1...v1.2.2

TEI is now Apache 2.0!

What's Changed

• Document how to send batched inputs by @osanseviero in https://github.com/huggingface/text-embeddings-inference/pull/222
• feat: add auto-truncate arg by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/224
• feat: add PredictPair to proto by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/225
• fix: fix auto_truncate for openai by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/228
• Change license to Apache 2.0 by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/231
• feat: Amazon SageMaker compatible images by @JGalego in https://github.com/huggingface/text-embeddings-inference/pull/103
• fix(CI): fix build all by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/236
• fix: fix cuda-all image by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/239
• Add SageMaker CPU images and validate by @philschmid in https://github.com/huggingface/text-embeddings-inference/pull/240

New Contributors

• @osanseviero made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/222
• @JGalego made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/103
• @philschmid made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/240

Full Changelog: https://github.com/huggingface/text-embeddings-inference/compare/v1.2.0...v1.2.1

What's Changed

• add cuda all image to facilitate deployment by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/186
• add splade pooling to Bert by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/187
• support vertex api endpoint by @drbh in https://github.com/huggingface/text-embeddings-inference/pull/184
• readme examples by @plaggy in https://github.com/huggingface/text-embeddings-inference/pull/180
• add_pooling_layer for bert classification by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/190
• add /embed_sparse route by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/191
• Applying Cargo.toml optimization options by @somehowchris in https://github.com/huggingface/text-embeddings-inference/pull/201
• Add Dockerfile-arm64 to allow docker builds on Apple M1/M2 architecture by @iandoe in https://github.com/huggingface/text-embeddings-inference/pull/209
• configurable payload limit by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/210
• add api_key for request authorization by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/211
• add all methods to vertex API by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/192
• add /decode route by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/212
• Input Types Compatibility with OpenAI's API (#112) by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/214

New Contributors

• @drbh made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/184
• @plaggy made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/180
• @somehowchris made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/201
• @iandoe made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/209

Full Changelog: https://github.com/huggingface/text-embeddings-inference/compare/v1.1.0...v1.2.0

Highlights

• Splade pooling

What's Changed

• Update Dockerfile to install curl by @jpbalarini in https://github.com/huggingface/text-embeddings-inference/pull/117
• fix loading of bert classification models by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/173
• splade pooling by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/174

New Contributors

• @jpbalarini made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/117

Full Changelog: https://github.com/huggingface/text-embeddings-inference/compare/v1.0.0...v.1.1.0

Highlights

• Support for Nomic models
• Support for Flash Attention for Jina models
• Metal backend for M* users
• /tokenize route to directly access the internal TEI tokenizer
• /embed_all route to allow client level pooling

What's Changed

• fix: limit the number of buckets for prom metrics by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/114
• feat: support flash attention for Jina by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/119
• feat: add support for Metal by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/120
• fix: fix turing for Jina and limit concurrency in docker build by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/121
• fix(router): fix panics on partial_cmp and empty req.texts by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/138
• feat(router): add /tokenize route by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/139
• feat(backend): support classification for bert by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/155
• feat: add embed_raw route to get all embeddings without pooling by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/154
• added docs for OTLP_ENDPOINT around the defaults and format sent by @MarcusDunn in https://github.com/huggingface/text-embeddings-inference/pull/157
• fix: use mimalloc to solve memory "leak" by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/161
• fix: remove modif of tokenizer by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/163
• fix: add cors_allow_origin to cli by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/162
• fix: use st max_seq_length by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/167
• feat: support nomic models by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/166

New Contributors

• @MarcusDunn made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/157

Full Changelog: https://github.com/huggingface/text-embeddings-inference/compare/v0.6.0...v1.0.0

What's Changed

• Doc build only if doc files were changed by @mishig25 in https://github.com/huggingface/text-embeddings-inference/pull/85
• fix: fix inappropriate title of API docs page by @ucyang in https://github.com/huggingface/text-embeddings-inference/pull/88
• fix: hf hub redirects by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/89
• feat: add grpc router by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/90
• fix: fix padding support in batch tokens by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/93
• fix: fix tokenizers with both whitespace and metaspace by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/96
• fix: enable http feature in http-builder by @zhangfand in https://github.com/huggingface/text-embeddings-inference/pull/98
• feat: add integration tests by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/101

New Contributors

• @mishig25 made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/85
• @ucyang made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/88
• @zhangfand made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/98

Full Changelog: https://github.com/huggingface/text-embeddings-inference/compare/v0.5.0...v0.6.0

What's Changed

• feat: accept batches in predict by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/78
• feat: rerank route by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/84

Full Changelog: https://github.com/huggingface/text-embeddings-inference/compare/v0.4.0...v0.5.0

What's Changed

• feat: USE_FLASH_ATTENTION env var by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/57
• docs: The initial version of the TEI docs for the hf.co/docs/ by @MKhalusova in https://github.com/huggingface/text-embeddings-inference/pull/60
• feat: support roberta by @kozistr in https://github.com/huggingface/text-embeddings-inference/pull/62
• fix: GH workflows update: added --not_python_module flag by @MKhalusova in https://github.com/huggingface/text-embeddings-inference/pull/66
• docs: Images links updated by @MKhalusova in https://github.com/huggingface/text-embeddings-inference/pull/72
• feat: add normalize option by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/70
• ci: Migrate CI to new Runners by @glegendre01 in https://github.com/huggingface/text-embeddings-inference/pull/74
• feat: add support for classification models by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/76

New Contributors

• @MKhalusova made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/60
• @kozistr made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/62
• @glegendre01 made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/74

Full Changelog: https://github.com/huggingface/text-embeddings-inference/compare/v0.3.0...v0.4.0

What's Changed

• feat: faster CPU image on AMD in https://github.com/huggingface/text-embeddings-inference/pull/35
• feat: support camembert in https://github.com/huggingface/text-embeddings-inference/pull/42
• feat: support float32 on cuda in https://github.com/huggingface/text-embeddings-inference/pull/41
• feat: support jinaAI variant in https://github.com/huggingface/text-embeddings-inference/pull/48

Full Changelog: https://github.com/huggingface/text-embeddings-inference/compare/v0.2.2...v0.3.0

What's Changed

fix: max_input_length should take into account position_offset (https://github.com/huggingface/text-embeddings-inference/commit/aec5efd77ee25ed3d6282eba1a51bae134c4815b)

Full Changelog: https://github.com/huggingface/text-embeddings-inference/compare/v0.2.1...v0.2.2

What's Changed

• fix: only use position offset for xlm-roberta (https://github.com/huggingface/text-embeddings-inference/commit/8c507c3531c7f0e7737936005653267e54228cd6)

Full Changelog: https://github.com/huggingface/text-embeddings-inference/compare/v0.2.0...v0.2.1

What's Changed

• add support for XLM-RoBERTa in https://github.com/huggingface/text-embeddings-inference/pull/5
• get number of tokenization workers from the number of CPU cores in https://github.com/huggingface/text-embeddings-inference/pull/8
• prefetch batch in https://github.com/huggingface/text-embeddings-inference/pull/10
• support loading from .pth in https://github.com/huggingface/text-embeddings-inference/pull/12
• add --pooling arg in https://github.com/huggingface/text-embeddings-inference/pull/14
• fix compute cap matching in https://github.com/huggingface/text-embeddings-inference/pull/21

Full Changelog: https://github.com/huggingface/text-embeddings-inference/compare/v0.1.0...v0.2.0

• No compilation step
• Dynamic shapes
• Small docker images and fast boot times. Get ready for true serverless!
• Token based dynamic batching
• Optimized transformers code for inference using Flash Attention, Candle and cuBLASLt
• Safetensors weight loading
• Production ready (distributed tracing with Open Telemetry, Prometheus metrics)