quantize_config.json instead of GPTQ_BITS env variables https://github.com/huggingface/text-generation-inference/pull/671bigcode/starcoder was an example https://github.com/huggingface/text-generation-inference/pull/661Full Changelog: https://github.com/huggingface/text-generation-inference/compare/v0.9.3...v0.9.4
Fetched April 7, 2026