# What changed in transformers > 149 changes in transformers 4 to 5 from its official release notes, upgrade guides and advisories: 34 breaking, 16 deprecations, 4 security, 47 new APIs and options. Each one cited. Sources checked 2026-10-27. HTML: https://unstale.sh/pypi/transformers/ To see which of these apply to a repository, run `npx @unstale/cli init` in it. ## transformers 5 ### Breaking changes - ColwiseParallel and RowwiseParallel now apply only to linear layers; custom tensor-parallel plans must use the dedicated embedding style for embeddings. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Every MoE tensor- and expert-parallel plan now shards experts; custom or downstream plans must add expert sharding to avoid keeping all experts on every rank. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Vision image processors now composite PNG transparency onto white rather than exposing black pixels. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - The minimum supported Python version is raised to 3.11. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Fixes from_pretrained on torch 2.7; distributed checkpointing requires torch>=2.8. (5.20.1) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.1) - Removes deprecated cache checks and methods; code calling them must switch to the current cache API. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Limits the Hub rotary kernel to CUDA inference because its backward pass produced incorrect gradients. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Raises the minimum kernels version to 0.17.2, enabling failed kernel loads to fall back to the original implementation with use_fallback=KernelizeFallback.ALL. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Entity special tokens such as are now treated as special tokens during tokenization and decoding, which may change output involving these tokens. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Removes accidental cross-model imports; imports of those names through model modules must instead come from their original modules. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Selects the query box with the highest objectness score instead of the OWL-ViT heuristic, so image-guided query embeddings and detections may differ. (5.19.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.19.0) - Deprecated pipelines now redirect, while removed pipelines raise an error. (5.19.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.19.0) - Allows prefix_allowed_tokens_fn to override model -inf and raises an exception when generation constraints are unsatisfiable. (5.18.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.18.0) - The distinction between Fast and Slow tokenizers is removed as Transformers focuses on the tokenizers backend. (5.0.0) — [upgrade guide](https://huggingface.co/blog/transformers-v5) - Flax and TensorFlow support is sunset in favor of PyTorch as the sole backend. (5.0.0) — [upgrade guide](https://huggingface.co/blog/transformers-v5) - Image processors exist only with their fast variant, which depends on the torchvision backend. (5.0.0) — [upgrade guide](https://huggingface.co/blog/transformers-v5) ### Security - Adds a linear-attention cache getter for recording-past state, avoiding direct exposure of cache internals. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Adds sanitization of chat input. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - huggingface/transformers: Arbitrary Code Execution During Model Initialization in the LightGlue Model Loading Path (5.5.0) — [security advisory](https://github.com/advisories/GHSA-FGCW-684Q-JJ6R) - HuggingFace transformers vulnerable to remote code execution (5.3.0) — [security advisory](https://github.com/advisories/GHSA-29PF-2H5F-8G72) ### Deprecations - Deprecates old input names. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Assignment to rope_scaling is deprecated; legacy RoPE keys are normalized. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Deprecates the unused get_uniform_frame_indices helper. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - The prefix is deprecated for SDPA and flash attention; use the regular attention implementation for continuous batching. Eager still requires paged|eager. (5.19.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.19.0) - Deprecates min-max pixels. (5.18.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.18.0) - Deprecates the use_mamba_kernels config flag because it no longer has any effect. (5.18.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.18.0) ### New APIs and options - Adds access to convolution states in linear-attention caches. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Adds expert_dtype, defaulting to fp8, with fp4 available so DeepSeek-V4.1 experts are not allocated as FP8. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Adds support for Recursive's mixture-of-experts language model. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Adds a linear-attention cache getter for recurrent state. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Adds support for Gradium's on-device text-to-speech models. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Adds support for a document-understanding model specialized for reading text-dense images. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Adds a tensor-parallel plan for Nemotron. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Adds Kyutai's text-to-speech model, which can run faster than real time on CPU and supports voice prompts in several forms. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Adds the Pocket TTS audio tokenizer, whose encoder and decoder can also be loaded independently. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Adds an fp8 backend that stores KV states in torch.float8_e4m3fn with a single scale per tensor. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Adds support for visual document understanding and retrieval with UltraVBert. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Adds support for Beam, a token-efficient model focused on coding and agentic use cases that can run on a team's own hardware, infrastructure, or cloud. (5.20.1) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.1) - Adds native Ling 3.0 Flash support through BailingHybridForCausalLM, so it loads without trust_remote_code=True. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Adds a unified quantization API for exporting with Dynamo, ONNX, OpenVINO, and ExecuTorch, including static and dynamic quantization and INT8/INT4 weight-only compression. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Adds support for Aleph Alpha's Kolibri 1 mixture-of-experts reasoning model. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Adds MiniCPM-V 4.7 support, including canvas M-RoPE, which assigns visual tokens temporal, height, and width positions. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Adds an audio model for long-form multi-speaker transcription, diarization, and timestamps, with chunking and hotword biasing support. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Adds native support for the multimodal Muse Spark Open model, which accepts text, images, audio, and video and generates text. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Centralizes sub-config initialization; custom or existing nested configs may need to declare sub-configs through this mechanism. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Adds bidirectional UltraBERT text encoders for classification and retrieval, including encoders supporting sequences up to 32,768 tokens. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Adds a 3B-parameter OCR model with gundam and base modes, long-form transcription, and optional region detections with bounding boxes. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Adds a multimodal embedding model that encodes text, images, audio, and video into a shared vector space, with configurable embedding dimensions and modality token budgets. (5.19.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.19.0) - Adds a token-dispatch implementation selected through this plan rule; it is now the default for Qwen3 MoE and Mellum and removes the requirement that EP size equal TP size. (5.19.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.19.0) - Adds support for registering new processing backends. (5.19.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.19.0) - Adds a Trainer field declaring whether compute_loss already scales for gradient accumulation. (5.19.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.19.0) - Adds support for loading GGUF models served with the Chat CLI. (5.18.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.18.0) - Adds the GTE BERT-style bidirectional encoder architecture, supporting multiple GTE checkpoint families and Snowflake's snowflake-arctic-embed-m-v2.0. (5.18.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.18.0) - Adds a vision-language model supporting text, image, and video inputs, with chain-of-thought reasoning via built-in thinking tokens. (5.18.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.18.0) - Adds support for MoE in the GGUF integration. (5.18.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.18.0) - Adds an open-weight streaming speaker diarization model supporting streaming and offline inference, up to eight speakers, configurable latency profiles, and configurable output frame resolution. (5.18.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.18.0) - Adds a multimodal reasoning model that reasons jointly over text, images, video, and sound. (5.18.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.18.0) - Adds the OpenVINO HF Exporter. (5.18.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.18.0) - Passes sampling_rate to the apply_chat_template call. (5.18.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.18.0) - Adds Trainer.end. (5.18.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.18.0) - A centralized abstraction for attention methods is introduced; non-eager methods such as FA1/2/3, FlexAttention, and SDPA move to the interface. (5.0.0) — [upgrade guide](https://huggingface.co/blog/transformers-v5) - Support for continuous batching and paged attention mechanisms is shipped for inference. (5.0.0) — [upgrade guide](https://huggingface.co/blog/transformers-v5) - GGUF files can be loaded in Transformers for further fine-tuning, and Transformers models can be converted to GGUF for llama.cpp. (5.0.0) — [upgrade guide](https://huggingface.co/blog/transformers-v5) - A Transformers-specific serving system is introduced that deploys an OpenAI API-compatible server. (5.0.0) — [upgrade guide](https://huggingface.co/blog/transformers-v5) ### Performance - Avoids a host-to-device copy in eager_mask. (5.20.2) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.2) - Speeds up Whisper generation on TPU by reducing device synchronizations and recompilation. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Encodes multimodal data only once during generation. (5.18.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.18.0) - Kernels are packaged so they are automatically used when the hardware and software permit it. (5.0.0) — [upgrade guide](https://huggingface.co/blog/transformers-v5) ### Behavior and default changes - Cache continuation is now supported for chunked prefill. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - On ROCm, flash_attention_2 now falls back to the AITER kernel when flash-attn is not installed. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - FSDP × TP model and optimizer checkpoints can now be saved and loaded across different parallel configurations. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Now accepts whole-number floats for integer tool arguments. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Image decoding now uses torchcodec. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Continuous batching fuses index-based and block-table cache updates into a single call, affecting custom code that calls the separate update paths. (5.19.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.19.0) - Regular flash and SDPA attention functions now support continuous batching directly; paged implementations for these are redirected to them. (5.19.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.19.0) - Initializes each layer from its own configuration, supporting heterogeneous sliding windows, attention chunks, convolution states, and attention head counts. (5.19.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.19.0) - Initializes each layer from its own configuration, supporting heterogeneous sliding windows, attention chunks, convolution states, and attention head counts. (5.19.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.19.0) - The Trainer now works with expert parallelism. (5.19.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.19.0) - Weight loading changes to make quantization a first-class citizen, with the stated goal of compatibility with major features and reliable training and inference. (5.0.0) — [upgrade guide](https://huggingface.co/blog/transformers-v5) - The tokenizers backend becomes the main tokenization backend; Sentencepiece and MistralCommon-backed tokenizers remain supported as non-default alternatives. (5.0.0) — [upgrade guide](https://huggingface.co/blog/transformers-v5) ### Fixes and other changes - Fixes preservation of background_color across the DeepSeek-VL and Janus image processors. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Fixes casting of dict-like inputs so generation no longer drops their type, including for ModelOutput. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Fixes image assignment during batched generation. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Fixes generation issues, including image placeholder handling in batched or beam-search generation. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Audio generation now respects generation_mode. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Fixes multi-zoom crops and related bugs in the mask-generation pipeline. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Prevents empty frame sampling for short videos. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Fixes Flash Attention compatibility with static caches and prevents BLOOM crashes with static cache or compiled generation. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Fixes a hardcoded torch.float64 use that caused crashes on MPS. (5.21) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.21) - Fixes zero expert gradients in bfloat16 when expert parallelism is used with ep_size < fsdp_size. (5.20.2) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.2) - Declares the input modalities of VLMs that report text only. (5.20.2) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.2) - Preserves input types in generation preparation, preventing problems with multimodal models. (5.20.2) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.2) - Fixes a small bug in auto mappings. (5.20.1) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.1) - Fixes DeepGEMM Mega MoE under expert parallelism. (5.20.1) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.1) - Works on torch 2.6 and 2.7. (5.20.1) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.1) - Fixes image assignment during batched generation. (5.20.1) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.1) - Fixes model loading on older torch versions. (5.20.1) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.1) - Restores behavior where enable_expert_parallel applies the expert-parallel plan alone. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Molmo2 image and video keyword arguments now follow the library's standard conventions. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Fixes residual flushing for multi-token updates. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Fixes static-cache generation with host-side inputs, nested inputs, and tensor parallelism. (5.20.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.20.0) - Fixes chunked prefill with multi-axis position ids for Qwen3-VL. (5.19.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.19.0) - Adds video to the model's input modalities. (5.19.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.19.0) - Fixes handling of seed=0. (5.19.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.19.0) - Fixes the source rows used for encoder repetition penalties after batch expansion. (5.19.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.19.0) - Preserves masked EOS scores when applying the penalty. (5.19.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.19.0) - No longer mutates the user's cache_config when handling quantized caches. (5.19.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.19.0) - Repairs cache reordering for quantized layers. (5.19.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.19.0) - MoE models that compute router logits now return them when output_router_logits=True; read them from the new output classes. (5.19.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.19.0) - Fixes repeated-ngram counting. (5.19.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.19.0) - Fixes AutoImageProcessor requiring torchvision when only Pillow is installed. (5.18.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v5.18.0) ## transformers 4 ### Breaking changes - Auto-tokenizers now return Rust tokenizers by default; set use_fast=False to obtain Python tokenizers. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - Replaced by past_key_values in BART-like, FSMT, and T5 models. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - Replaced by past_key_values in BART-like, FSMT, and T5 models. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - The argument is replaced by top_k. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - The tokenizer encoding argument is replaced by is_split_into_words. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - Removed in several language-model classes in favor of labels. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - Removed in several masked-language-model classes in favor of labels. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - The tokenizer attribute is replaced by model_max_length. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - Replaced by past_key_values in CTRL and GPT-2 models. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - The Transfo-XL modeling method is replaced by reset_memory_length. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - Model outputs are now named-tuples-like objects by default; set return_dict=False for tuple outputs, which can be unpacked. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - The tokenizer attribute is replaced by return_length. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - SentencePiece is no longer a required dependency, so tokenizers that depend on it are unavailable in a standard installation; install it separately or use transformers[sentencepiece]. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - The TFTrainer argument is removed in favor of args.prediction_loss_only. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - The Transfo-XL configuration attribute is replaced by tie_words_embeddings. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - The Trainer argument is removed in favor of args.prediction_loss_only. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - The Trainer argument is removed; use TensorBoardCallback(tb_writer=...) instead. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - Model files now reside in model-specific folders; import BertLayer from this path instead of transformers.modeling_bert. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) ### Deprecations - Deprecated in favor of prediction_loop. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - Deprecated in favor of run_model. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - Deprecated in favor of setup_wandb. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - Deprecated in favor of evaluation_strategy. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - Deprecated in favor of log. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - Deprecated in favor of prediction_loop. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - Deprecated in favor of training_step. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - Deprecated in favor of is_local_process_zero. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - Deprecated in favor of is_world_process_zero. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - Deprecated in favor of evaluation_strategy. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) ### New APIs and options - Experimental encoder-model templates generate model, configuration, tokenization, and test files and place and reference them in the appropriate locations. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - Multilingual T5 model support and pretrained checkpoints are added. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - Pipelines now include additional features out of the box, including grouped entities in token classification. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - A PrefixConstrainedLogitsProcessor is added. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - ProphetNet adds the prepare_seq2seq_batch function. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - T5 version 1.1 model support and pretrained checkpoints are added. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - TensorFlow Longformer adds SequenceClassification, MultipleChoice, and TokenClassification heads. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - DPR is now available in TensorFlow to match its PyTorch counterpart. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) - Install this extra to obtain the SentencePiece-dependent tokenizers alongside Transformers. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0) ### Fixes and other changes - The data_collator attribute should be a callable. (4.0.0) — [release notes](https://github.com/huggingface/transformers/releases/tag/v4.0.0)