unstale

pypi / sacrebleu

sacrebleu

What changed in sacrebleu 2, from its official release notes, upgrade guides and advisories. Sources checked 2026-10-25.

8 breaking14 new APIs and options2 performance1 behavior and defaults

See which of these apply to your code, ranked, in your agent:

npx @unstale/cli init

@unstale/cli 0.2.6 on npm

sacrebleu 225 changes

Breaking changes 8

  • Python 3.8 is no longer supported; Python 3.13 is added, with requires-python set to >=3.9.

    2.6.0release notes

  • When there are no n-gram matches at all, BLEU skips smoothing and returns 0.0.

    2.0.0release notes

  • Single-system CLI output now defaults to json instead of the old single-line output.

    2.0.0release notes

  • Inputs that do not follow the expected annotations now raise exceptions instead of being manipulated through isinstance checks.

    2.0.0release notes

  • All metrics are now scaled into the [0, 100] range.

    2.0.0release notes

  • Python versions below 3.6 are no longer supported.

    2.0.0release notes

  • Export this environment variable as text to persistently keep the old text output format.

    2.0.0release notes

  • The signature field separator is now | and key values are separated with : rather than ., removing the + separator.

    2.0.0release notes

New APIs and options 14

  • A new tokenizer, spBLEU-1K, is available.

    2.6.0release notes

  • This option disables effective order smoothing so scores match chrF++, Moses and NLTK implementations; effective ordering remains the default.

    2.0.0release notes

  • CHRF now supports multiple references.

    2.0.0release notes

  • The CLI exposes chrF+ support through this option.

    2.0.0release notes

  • Enables confidence interval estimation through bootstrap resampling for single-system evaluation.

    2.0.0release notes

  • Adds a format selector, with -f as a short flag.

    2.0.0release notes

  • The maximum n-gram order for BLEU can now be modified.

    2.0.0release notes

  • Metric-specific arguments are prefixed with --chrf and --ter, while BLEU argument names remain unchanged.

    2.0.0release notes

  • The CLI can evaluate multiple systems for one test set, using input files or a tab-separated stream on STDIN, and renders results in several formats.

    2.0.0release notes

  • Enables paired approximate randomization tests when evaluating multiple systems.

    2.0.0release notes

  • Enables paired bootstrap resampling when evaluating multiple systems.

    2.0.0release notes

  • Adds statistical significance testing for BLEU, chrF and TER.

    2.0.0release notes

  • TER arguments can now be modified through the CLI; TERCOM defaults are retained.

    2.0.0release notes

  • The CHRF word_order argument adds chrF+ support.

    2.0.0release notes

Performance 2

  • The intl tokenizer now uses the regex module and is faster on the noted test set.

    2.0.0release notes

  • Tokenizers now cache results, which seems to speed up evaluation.

    2.0.0release notes

Behavior and default changes 1

  • Tabular outputs in multi-system evaluation mode now use colored strings.

    2.0.0release notes