Hugging Face

Recently shipped TRL's fused LM head — up to 6.9× longer trainable sequences on the same GPU — and an AWS Trainium training path.

TRL fused its LM head and dropped its vendored loss module.

• DPO, KTO, GRPO, SFT, and Distillation score tokens in Triton tiles without building the [batch, seq, vocab] logits tensor, on by default.
• Max DPO sequence length on gemma-3-1b at 79 GiB rose from 10,240 to 59,392; KTO to 63,488.
• trl.losses is gone: the fused copies had drifted on GRPO grad_norm and KTO's PEFT reference model, and never all-reduced gradients under plain DDP.
• Training now runs on AWS Trainium, halving a 20-step Trainium2 Qwen3-0.6B SFT run from 260 s to 130 s.

Transformers v5.19 added a cross-modal embedder. EmbeddingGemma 2 encodes text, images, audio, and video into one 768-dimensional space, truncatable to 512, 256, or 128 dimensions, with unused vision or audio towers disposable at load. Nemotron 3 Diarization tracks up to eight speakers streaming or offline, from an 80 ms buffer to 30.4 s. MoE models carry a breaking change.

Diffusers 0.41 shipped Qwen-Image 2.1. One model for generation and editing: a 7B visual component, multiple reference images, native RGBA transparency, and LoRA training. Minor releases now cluster around model integrations; patches stay fix-only.

Datasets 5.1 widened format support. Vortex backs datasets with high-performance columnar files, Harbor covers RL environments, and FASTA, FASTQ, and GenBank handle DNA, protein, and molecular structure.

huggingface_hub 2.1 improved Jobs. Retries with attempts, reruns from a saved spec, cron rescheduling without recreating, and token-protected or public port exposure. The v2.0 breaking release stands: the HTTP stack moved to httpx2, so custom client factories and transport except clauses need httpx2 types.

Smaller surfaces. Tokenizers hit 1.0 RC with the same API and token IDs, a 6x smaller crate, and faster tokenization across languages. PEFT 0.21 added Riemannian-preconditioned LoRA and KaSA. TEI v1.9.4 added ARM64 and multi-arch CUDA images plus minimal AMD ROCm support.