# What changed in sacrebleu

> 25 changes in sacrebleu 2 from its official release notes, upgrade guides and advisories: 8 breaking, 14 new APIs and options. Each one cited. Sources checked 2026-10-25. HTML: https://unstale.sh/pypi/sacrebleu/

To see which of these apply to a repository, run `npx @unstale/cli init` in it.

## sacrebleu 2

### Breaking changes

- Python 3.8 is no longer supported; Python 3.13 is added, with requires-python set to >=3.9. (2.6.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.6.0)
- When there are no n-gram matches at all, BLEU skips smoothing and returns 0.0. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)
- Single-system CLI output now defaults to json instead of the old single-line output. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)
- Inputs that do not follow the expected annotations now raise exceptions instead of being manipulated through isinstance checks. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)
- All metrics are now scaled into the [0, 100] range. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)
- Python versions below 3.6 are no longer supported. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)
- Export this environment variable as text to persistently keep the old text output format. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)
- The signature field separator is now | and key values are separated with : rather than ., removing the + separator. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)

### New APIs and options

- A new tokenizer, spBLEU-1K, is available. (2.6.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.6.0)
- This option disables effective order smoothing so scores match chrF++, Moses and NLTK implementations; effective ordering remains the default. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)
- CHRF now supports multiple references. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)
- The CLI exposes chrF+ support through this option. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)
- Enables confidence interval estimation through bootstrap resampling for single-system evaluation. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)
- Adds a format selector, with -f as a short flag. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)
- The maximum n-gram order for BLEU can now be modified. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)
- Metric-specific arguments are prefixed with --chrf and --ter, while BLEU argument names remain unchanged. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)
- The CLI can evaluate multiple systems for one test set, using input files or a tab-separated stream on STDIN, and renders results in several formats. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)
- Enables paired approximate randomization tests when evaluating multiple systems. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)
- Enables paired bootstrap resampling when evaluating multiple systems. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)
- Adds statistical significance testing for BLEU, chrF and TER. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)
- TER arguments can now be modified through the CLI; TERCOM defaults are retained. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)
- The CHRF word_order argument adds chrF+ support. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)

### Performance

- The intl tokenizer now uses the regex module and is faster on the noted test set. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)
- Tokenizers now cache results, which seems to speed up evaluation. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)

### Behavior and default changes

- Tabular outputs in multi-system evaluation mode now use colored strings. (2.0.0) — [release notes](https://github.com/mjpost/sacrebleu/releases/tag/v2.0.0)