# What changed in promptfoo

> 107 changes in promptfoo 0.123 to 0.124 from its official release notes, upgrade guides and advisories: 23 breaking, 4 security, 17 new APIs and options. Each one cited. Sources checked 2026-10-19. HTML: https://unstale.sh/npm/promptfoo/

To see which of these apply to a repository, run `npx @unstale/cli init` in it.

## promptfoo 0.124

### Breaking changes

- Browser SDKs now require explicit installation. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Browser SDKs now require explicit installation. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- The Codex Security SDK is now opt-in rather than included by default. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- The Codex Security SDK is now opt-in rather than loaded by default. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- The hosted ChatKit provider has been removed. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- The hosted ChatKit provider was removed. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- The Langfuse SDK is now opt-in rather than included by default. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- The Langfuse SDK is now opt-in rather than loaded by default. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Local Transformers is now opt-in rather than included by default. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Local Transformers is now opt-in rather than loaded by default. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- OpenAI Agents now requires an explicit install. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- OpenAI Agents now requires explicit installation. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- The Slack SDK is now opt-in rather than included by default. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- The Slack SDK is now opt-in rather than loaded by default. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- The CLI now returns a failing exit code when a target error stops an evaluation. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Invalid VLSU concurrency values are now rejected. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- WatsonX SDKs are now opt-in rather than included by default. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- WatsonX SDKs are now opt-in rather than loaded by default. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)

### Security

- Red-team stops Meta on invalid JSON requests. (0.124.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.1)

### New APIs and options

- Exposes text-token pricing from the provider catalog. (0.124.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.1)
- Adds support for OpenAI Decisions. (0.124.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.1)
- Claude Opus 5.5 is now supported as a provider model. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Paused Claude server-tool turns can now be resumed. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Claude Sonnet 5.5 is now supported as a provider model. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Coding agents can now run in isolated workspaces. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- GPT-6 Sol and Luna are now supported provider models. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- GPT-6.1 Sol and its ultrafast variant are now supported. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Grok 4.7 is now supported as a provider model. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- OpenAI safety identifiers are now supported by providers. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- A TypeSafe Jev provider is now available. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)

### Performance

- Indexes extracted tests when filtering evaluation results. (0.124.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.1)

### Behavior and default changes

- Assertions can record cost without requiring a threshold. (0.124.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.1)
- Promptfoo now warns when Azure service principal configuration is partial. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Bedrock GPT-6 Sol and Luna are now treated as reasoning models. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Claude providers now report usage per call. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Metadata filtering now supports comma-separated OR values. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Redteam reports which turn the grader flagged. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- GPT-6 Chat is no longer capped by OPENAI_MAX_TOKENS. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)

### Fixes and other changes

- Fixes handling of assertion comments. (0.124.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.1)
- Preserves grading for asynchronous expressions. (0.124.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.1)
- Preserves Cloud errors. (0.124.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.1)
- Retains the prepared custom environment. (0.124.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.1)
- Fixes handling of dataset limits. (0.124.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.1)
- Samples dataset plugin rows uniformly. (0.124.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.1)
- Evaluation now respects the #gid in Google Sheets URLs. (0.124.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.1)
- Exposes incomplete Responses status from providers. (0.124.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.1)
- Preserves complete JSON objects. (0.124.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.1)
- Preserves grading for keyword properties. (0.124.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.1)
- Red-team runs report missing target output as errors. (0.124.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.1)
- Adds a check for npm aliases. (0.124.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.1)
- Code-scan OIDC tokens are minted after CLI installation. (0.124.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.1)
- Preserves regex operands after yield. (0.124.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.1)
- Array sampling no longer has bias. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Assertions now accept grader result shapes that earlier releases recorded. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Bedrock providers now honor region overrides. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Blank BLEU references are ignored. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Vertex now preserves zero Claude sampling parameters. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- CommonJS consumers can now use the package API. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Custom grading results are now validated. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Evaluation configs downloaded from the web UI remain importable. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- ElevenLabs TTS now respects the configured cache policy. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- The --env-file option and config environment are now honored for prompt separator and assertion concurrency. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Configuration now accepts numeric and boolean environment values. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- A literal comma is now allowed in metadata filter values. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Inverse RAG scores are clamped to the documented 0–1 range. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- The not- prefix is now honored for model-graded-closedqa and factuality assertions. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Null Ollama tool arguments are now treated as empty. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Ollama tool calls are accepted back in multi-turn conversations. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- OpenAI Agents token counts are now validated. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Providers now honor cache overrides supplied per call. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Redteam now preserves zero sampling values. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Iterative, crescendo, and iterative:tree now honor an explicitly configured redteamProvider. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)
- Vercel provider response handling was simplified. (0.124.0) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.124.0)

## promptfoo 0.123

### Breaking changes

- credit_balance_exhausted is now treated as a hard quota error. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Ollama embeddings now use /api/embed and honor configuration. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- RAG assertion thresholds now default to 0.5. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Inverting a search-rubric assertion now inverts its score along with its pass result. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Non-string search rubrics are now rejected. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)

### Security

- Agents endpoint credentials are isolated. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Custom header requests have been hardened. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Local configuration is no longer leaked into provider request bodies. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)

### New APIs and options

- A Gemini 3.8 provider is available for use in evaluations. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Providers now support GPT-Live voice sessions. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- A provider for the OpenAI Agents API is now available. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Portable HTTP configuration schemas are available for providers. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Portable MCP configuration schemas are available for providers. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Vertex Live is available as a provider. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)

### Behavior and default changes

- Gemini provider tool and media support has been updated. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Providers support Ollama 0.34 features. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Ollama provider results now surface thinking output and the finish reason. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)

### Fixes and other changes

- Image loading now supports AVIF and TIFF, and SDK options are honored. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- The normalized Envoy base URL is applied after the user configuration spread. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- MCP now accepts fine-tuned model identifiers. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Foundry overrides are now honored. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Prompt Gemini tool policy now takes precedence over passthrough. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- The Helicone key now honors --env-file and config env. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- The Langfuse base URL now honors --env-file and config env. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Default MCP enablement is preserved. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Grader failures are no longer inverted for not-classifier assertions. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Grader failures are no longer inverted for not-search-rubric assertions. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Nova Sonic providers now honor inference configuration. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Ollama HTTP errors are surfaced instead of being returned as empty output. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Streamed Ollama tool calls are now preserved. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- CLI result export now handles large values. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Snowflake provider configuration is now forwarded. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Resolved TogetherAI model parameters are no longer overwritten. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)
- Malformed webhook results are now rejected. (0.123.1) — [release notes](https://github.com/promptfoo/promptfoo/releases/tag/0.123.1)