chore(deps): update dependency promptfoo to ^0.90.0

[mend-for-github-com]

This PR contains the following updates:

Package	Type	Update	Change
promptfoo	dependencies	minor	^0.32.0 -> ^0.90.0

By merging this PR, the below vulnerabilities will be automatically resolved:

Severity	CVSS Score	CVE
Medium	5.3	CVE-2024-47764
Medium	5.3	CVE-2024-47764

---

Release Notes

promptfoo/promptfoo (promptfoo)

v0.90.0

Compare Source

What's Changed

Features

• cli: Added 'pf' as an alias for the 'promptfoo' command (@​mldangelo, #​1745)
• providers(bedrock): Added support for AI21 Jamba Models and Meta Llama 3.1 Models (@​mldangelo, #​1753)
• providers(python): Added support for file:// syntax for Python providers (@​mldangelo, #​1748)
• providers(http): Added support for raw requests (@​typpo, #​1749)
• cli: implement cloud Login functionality for private sharing (@​sklein12, #​1719)
• cli(redteam): aliased 'eval' in redteam namespace and prioritized redteam.yaml over promptfooconfig.yaml (@​typpo, #​1664)

Chores

• providers(http): Added templating support for provider URLs (@​mldangelo, #​1747)
• cli: read config files from directory paths (@​andretran, #​1721)
• Added PROMPTFOO_EXPERIMENTAL environment variable (@​typpo)
• Simplified redteam consent process (@​typpo)
• Improved input handling for login prompts (@​mldangelo)
• Updated dependencies (@​mldangelo)

Fixes

• webui: fix route to edit eval description(@​sklein12, #​1754)
• cli: prevent logging of empty output paths (@​mldangelo)

Documentation

• Added raw HTTP request example (@​typpo)
• Updated documentation to prefer prebuilt versions (@​sklein12, #​1752)

CI/CD

• Triggered release step in nexe build for tagged branches (@​mldangelo)
• Updated release token in GitHub Actions workflow (@​mldangelo)
• Added continue-on-error to nexe-build job (@​mldangelo)

New Contributors

• @​andretran made their first contribution in #​1721

Full Changelog: promptfoo/promptfoo@0.89.4...0.90.0

v0.89.4

Compare Source

What's Changed

• feat(webui): display suggestions by @​typpo in https://github.com/promptfoo/promptfoo/pull/1739
• feat: headless browser provider by @​typpo in https://github.com/promptfoo/promptfoo/pull/1736
• feat: suggestions by @​typpo in https://github.com/promptfoo/promptfoo/pull/1723
• feat: improvements to http and websocket providers by @​typpo in https://github.com/promptfoo/promptfoo/pull/1732
• fix(prompts): add handling for function prompt by @​albertlieyingadrian in https://github.com/promptfoo/promptfoo/pull/1724
• fix: empty state for webui by @​typpo in https://github.com/promptfoo/promptfoo/pull/1727
• chore: add costs for OpenAI model "gpt-4o-2024-08-06" by @​skarlcf in https://github.com/promptfoo/promptfoo/pull/1728
• fix: catch errors when creating share url by @​typpo in https://github.com/promptfoo/promptfoo/pull/1726https://github.com/promptfoo/promptfoo/pull/1725
• fix: add missing outputPath by @​albertlieyingadrian in https://github.com/promptfoo/promptfoo/pull/1734
• fix: output path when PROMPTFOO_LIGHTWEIGHT_RESULTS is set by @​typpo in https://github.com/promptfoo/promptfoo/pull/1737
• chore: Move share action to server by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1743
• docs: Update documentation for Tree-based Jailbreaks Strategy by @​vingiarrusso in

New Contributors

• @​vingiarrusso made their first contribution in https://github.com/promptfoo/promptfoo/pull/1725
• @​skarlcf made their first contribution in https://github.com/promptfoo/promptfoo/pull/1728

Full Changelog: promptfoo/promptfoo@0.89.3...0.89.4

v0.89.3

Compare Source

What's Changed

• Bug Fixes:

  • Improved sanitization of generations (#​1713 by @​typpo)
  • Reverted config changes to resolve prompt file bug (#​1722 by @​mldangelo)

• Docs

  • Added more information to the enterprise page (#​1714 by @​typpo)
  • Updated the about page (#​1715 by @​typpo)
  • Minor landing page updates (#​1718 by @​typpo)

Full Changelog: 0.89.2...0.89.3

v0.89.2

Compare Source

Release Notes

Version 0.89.2

Improvements

• Dependencies: Updated project dependencies (@​mldangelo)
• Website: Added truncate functionality to the site (@​typpo)

Fixes

• Fixed Node cache dependency issue (@​typpo)

Chore

• Improved nexe build workflow artifact handling in CI pipeline (@​mldangelo)

Miscellaneous

• Bumped version to 0.89.2 (@​typpo)

Full Changelog: promptfoo/promptfoo@0.89.1...0.89.2

v0.89.1

Compare Source

What's Changed

Features

• feat: Redteam dashboard by @​typpo in #​1709
• feat(provider/openai): support loading response_format from a file by @​albertlieyingadrian in #​1711
• feat: add WebSocket provider by @​typpo in #​1712
• feat(matchers): add external file loader for LLM rubric by @​albertlieyingadrian in #​1698

Documentation

• docs: GPT vs O1 guide by @​typpo in #​1703

Chores

• chore(deps): bump openai from 4.61.1 to 4.62.0 by @​dependabot in #​1706
• chore(deps): bump @azure/openai-assistants from 1.0.0-beta.5 to 1.0.0-beta.6 by @​dependabot in #​1707

---

Full Changelog: 0.89.0...0.89.1

v0.89.0

Compare Source

What's Changed

Features

• feat: Migrate NextUI to a React App (#​1637) by @​sklein12
• feat: add golang provider (#​1693) by @​typpo
• feat(util): add nunjucks template support for file path (#​1688) by @​albertlieyingadrian
• feat: make config prompts optional (#​1694) by @​typpo
• feat(redteam): top level targets, plugins, strategies (#​1689) by @​typpo

Fixes

• fix(python): change PythonShell mode to binary to fix unicode encoding issues (#​1671) by @​mldangelo
• fix(python): check --version for executable path validation (#​1690) by @​mldangelo
• fix(providers): Mistral Error Reporting (#​1691) by @​GICodeWarrior

Chores

• chore(redteam): plumb scores per plugin and strategy (#​1684) by @​typpo
• chore(redteam): redteam init indent plugins and strategies by @​typpo
• chore(redteam): redteam onboarding updates (#​1695) by @​typpo
• chore(redteam): update some framework mappings by @​typpo
• chore(deps): bump openai from 4.61.0 to 4.61.1 (#​1696) by @​dependabot
• chore(deps): remove nexe dev dependency by @​mldangelo
• chore(deps): update eslint and related packages by @​mldangelo

Refactors

• refactor(csv): improve assertion parsing and add warning for single underscore usage (#​1692) by @​mldangelo

Documentation

• docs: improve Python provider example with stub LLM function by @​mldangelo

Full Changelog: promptfoo/promptfoo@0.88.0...0.89.0

v0.88.0

Compare Source

What's Changed

✨ Features

• Added support for file:// prefix for local file paths in the tests: field in configuration (@​mldangelo)
• Added --env-path as an alias for the --env-file option in CLI (@​mldangelo)
• Introduced PROMPTFOO_LIGHTWEIGHT_RESULTS environment variable to optimize result storage (@​typpo)
• Added validatePythonPath function and improved error handling for Python scripts (@​mldangelo)
• Displayed custom named scores in the Web UI (@​khp)
• Improved support for structured outputs in the OpenAI provider (@​mldangelo)
• Added OpenAI Assistant's token usage statistics (@​albertlieyingadrian)
• Added pricing information for Azure OpenAI models (@​mldangelo)
• Improved API URL formatting for Azure OpenAI provider (@​mldangelo)

🐛 Bug Fixes

• Fixed prompt normalization when reading configurations (@​mldangelo)
• Resolved Docker image issues by adding Python, ensuring the next output directory exists, and disabling telemetry (@​mldangelo)
• Improved message parsing for the Anthropic provider (@​mldangelo)
• Fixed error in loading externally defined OpenAI function calls (@​mldangelo)
• Corrected latency assertion error for zero milliseconds latency (@​albertlieyingadrian)

📚 Documentation

• Added a new Red Team introduction and case studies to the documentation (@​typpo)
• Updated model references and default LLM models in the documentation (@​mldangelo)
• Fixed typos and broken image links in the documentation (@​mldangelo, @​typpo)

🔨 Refactoring

• Refactored Red Team commands and types to improve code organization (@​mldangelo)
• Moved evaluateOptions initialization to evalCommand (@​mldangelo)
• Centralized cost calculation logic in providers (@​mldangelo)

New Contributors

• @​khp made their first contribution in #​1669

Full Changelog: promptfoo/promptfoo@0.87.1...0.88.0

v0.87.1

Compare Source

What's Changed

Chores

• Add OpenAI o1 pricing by @​typpo in #​1649
• Add support for OpenAI o1 max completion tokens by @​mldangelo in #​1650

Bug Fixes

• Share link issue when self-hosting by @​typpo in #​1647
• Fix OpenAI function tool callbacks handling by @​mldangelo in #​1648

Documentation

• Fix broken anchor links by @​mldangelo in #​1645
• Add documentation for Echo provider by @​mldangelo in #​1646

Full Changelog: v0.87.0...v0.87.1

v0.87.0

Compare Source

What's Changed

• feat: remote strategy execution by @​typpo in https://github.com/promptfoo/promptfoo/pull/1592
• fix: run db migrations first thing in cli by @​typpo in https://github.com/promptfoo/promptfoo/pull/1638
• chore: add --remote to eval by @​typpo in https://github.com/promptfoo/promptfoo/pull/1639
• chore(deps-dev): bump @​aws-sdk/client-bedrock-runtime from 3.649.0 to 3.650.0 by @​dependabot in https://github.com/promptfoo/promptfoo/pull/1640
• chore(deps): bump openai from 4.58.2 to 4.59.0 by @​dependabot in https://github.com/promptfoo/promptfoo/pull/1641
• chore: ability to record when feature is used by @​typpo in https://github.com/promptfoo/promptfoo/pull/1643
• site: intro and image updates by @​typpo in https://github.com/promptfoo/promptfoo/pull/1636

Full Changelog: promptfoo/promptfoo@0.86.1...0.87.0

v0.86.1

Compare Source

What's Changed

• feat: cross-session leak plugin by @​typpo in https://github.com/promptfoo/promptfoo/pull/1631
• fix: quickswitcher by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1635

Full Changelog: promptfoo/promptfoo@0.86.0...0.86.1

v0.86.0

Compare Source

What's Changed

Features

• feat: Added MITRE Atlas plugin aliases by @​typpo in #​1629
• chore: Removed the NextAPI by @​sklein12 in #​1599

Bug Fixes

• fix: Improved rate limiting handling by @​sinedied in #​1633
• fix: Ensured name:value pairs are unique, rather than just names, for tags by @​sklein12 in #​1621

Chores

• chore(deps-dev): Bumped @aws-sdk/client-bedrock-runtime from 3.645.0 to 3.649.0 by @​dependabot in #​1632
• chore: Fixed paths for ts-node by @​sklein12 in #​1628
• chore: Standardized paths by @​sklein12 in #​1627
• chore(deps): Bumped @anthropic-ai/sdk from 0.27.2 to 0.27.3 by @​dependabot in #​1625
• chore(deps): Bumped openai from 4.58.1 to 4.58.2 by @​dependabot in #​1624

New Contributors

• @​sinedied made their first contribution in #​1633

Full Changelog: 0.85.2...0.86.0

v0.85.2

Compare Source

What's Changed

• feat: compliance status in redteam reports by @​typpo in https://github.com/promptfoo/promptfoo/pull/1619
• fix: prompt parsing by @​typpo in https://github.com/promptfoo/promptfoo/pull/1620

Full Changelog: promptfoo/promptfoo@0.85.1...0.85.2

v0.85.1

Compare Source

What's Changed

• feat: add support for markdown prompts by @​mldangelo in https://github.com/promptfoo/promptfoo/pull/1616
• chore(deps): bump openai from 4.58.0 to 4.58.1 by @​dependabot in https://github.com/promptfoo/promptfoo/pull/1617
• fix: Indirect Prompt Injection missing purpose and will only generate… by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1618

Full Changelog: promptfoo/promptfoo@0.85.0...0.85.1

v0.85.0

Compare Source

What's Changed

Features

• feat(mistral): Update chat models and add embedding provider by @​mldangelo in #​1614
• feat(templates): Allow Nunjucks templating in grader context by @​mldangelo in #​1606
• feat(redteam): Add remote generation for multilingual strategy by @​mldangelo in #​1603
• feat(redteam): ASCII smuggling plugin by @​typpo in #​1602
• feat(redteam): More direct prompt injections by @​typpo in #​1600
• feat(redteam): Prompt injections for all test cases by @​typpo in commit 28605413

Fixes

• fix(redteam): Correct metric name for misinfo/pii/etc plugins by @​typpo in #​1605
• fix(redteam): Remove quotes and numbered results from generated prompts by @​typpo in #​1601
• fix(redteam): Move purpose to the right place in redteam template by @​typpo in commit 00b2ed1c

Refactors

• refactor: Improve project initialization and error handling by @​mldangelo in #​1591

Chores

• chore(deps): Bump openai from 4.57.3 to 4.58.0 by @​dependabot in #​1608
• chore(deps): Bump openai from 4.57.2 to 4.57.3 by @​dependabot in #​1594
• chore: Warn if API keys are not present when running promptfoo init by @​cristiancavalli in #​1577
• chore: Add info to contains-all and icontains-all error by @​typpo in #​1596
• chore(redteam): Export graders by @​sklein12 in #​1593
• chore(redteam): Export prompt generators by @​sklein12 in #​1583

Documentation

• docs: Add information on loading scenarios from external files by @​mldangelo in commit ddcc6e59
• docs(redteam): Red team introduction by @​typpo in commit ba5fe14c and commit 60624456
• docs(redteam): Minor redteam update by @​typpo in commit 7cad8da5

Tests

• test(redteam): Enhance nested quotes handling in parseGeneratedPrompts by @​mldangelo in commit 36f6464a

New Contributors

• @​cristiancavalli made their first contribution in #​1577

---

Full Changelog: promptfoo/promptfoo@0.84.1...0.85.0

v0.84.1

Compare Source

What's Changed

• fix: json parsing infinite loop by @​typpo in https://github.com/promptfoo/promptfoo/pull/1590
• fix: add cache and timeout to remote grading by @​typpo in https://github.com/promptfoo/promptfoo/pull/1589

Full Changelog: promptfoo/promptfoo@0.84.0...0.84.1

v0.84.0

Compare Source

What's Changed

Features

• Support for remote llm-rubric (@​typpo in #​1585)

Bug Fixes

• Resolve foreign key constraint in deleteAllEvals (@​mldangelo in #​1581)
• Don't set OpenAI chat completion seed=0 by default (@​Sasja in #​1580)
• Improve strategy JSON parsing (@​typpo in #​1587)
• Multilingual strategy now uses redteam provider (@​typpo in #​1586)
• Handle redteam remote generation error (@​typpo)
• Redteam refusals are not failures for Vertex AI (@​typpo)

Refactoring

• Reorganize redteam exports and add Strategies (@​mldangelo in #​1588)

Documentation

• Update OpenAI config documentation (@​mldangelo)
• Improve Azure environment variables and configuration documentation (@​mldangelo)

Chores

• Bump dependencies and devDependencies (@​mldangelo)
• Set stream: false in Ollama provider (@​typpo, #​1568)
• Bump openai from 4.57.0 to 4.57.1 (@​dependabot in #​1579)
• Regenerate JSON schema based on type change (@​mldangelo)
• Synchronize EnvOverrides in types and validators (@​mldangelo)

New Contributors

• @​Sasja made their first contribution in #​1580

Full Changelog: promptfoo/promptfoo@0.83.2...0.84.0

v0.83.2

Compare Source

What's Changed

Full Changelog: promptfoo/promptfoo@0.83.1...0.83.2

v0.83.1

Compare Source

Full Changelog: promptfoo/promptfoo@0.83.0...0.83.1

v0.83.0

Compare Source

What's Changed

• feat: add onboarding flow for http endpoint by @​typpo in https://github.com/promptfoo/promptfoo/pull/1572
• feat: remote generation on the cli by @​typpo in https://github.com/promptfoo/promptfoo/pull/1570
• docs: update YAML syntax for prompts and providers arrays by @​mldangelo in https://github.com/promptfoo/promptfoo/pull/1574

Full Changelog: promptfoo/promptfoo@0.82.0...0.83.0

v0.82.0

Compare Source

What's Changed

Features

• feat(redteam): add remote generation for purpose and entities by @​mldangelo
• feat: add delay option for redteam generate and refactor plugins by @​typpo

Bug Fixes

• fix: validate all plugins before running any by @​typpo
• fix: remove indirect prompt injection config.systemPrompt dependency by @​typpo
• fix: show all strategies on report by @​typpo
• fix: bfla grading by @​typpo

Chores and Refactoring

• chore: simplify redteam types by @​typpo
• chore: move redteam command locations by @​typpo
• chore: defaults for redteam plugins/strategies by @​typpo
• chore: clean up some redteam onboarding questions by @​typpo
• chore: export redteam plugins by @​typpo
• chore: rename envar by @​typpo
• chore: add PROMPTFOO_NO_REDTEAM_MODERATION envar by @​typpo
• chore(redteam): add progress bar to multilingual strategy by @​mldangelo
• chore(redteam): export extraction functions by @​mldangelo
• chore(docker): install peer dependencies during build by @​mldangelo
• docs: update file paths to use file:// prefix by @​mldangelo

Dependencies

• chore(deps-dev): bump @​aws-sdk/client-bedrock-runtime from 3.637.0 to 3.642.0 by @​dependabot
• chore(deps): bump replicate from 0.32.0 to 0.32.1 by @​dependabot
• chore(deps): bump openai from 4.56.1 to 4.57.0 by @​dependabot
• chore(deps): bump the github-actions group with 2 updates by @​dependabot

Full Changelog: promptfoo/promptfoo@0.81.5...0.82.0

v0.81.5

Compare Source

What's Changed

• feat: global env var in templates by @​typpo in https://github.com/promptfoo/promptfoo/pull/1553
• fix: harmful grader by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1554
• chore: include createdAt in getStandaloneEvals by @​typpo in https://github.com/promptfoo/promptfoo/pull/1550
• chore: write eval tags to database and add migration by @​typpo in https://github.com/promptfoo/promptfoo/pull/1551
• style: enforce object shorthand rule by @​mldangelo in https://github.com/promptfoo/promptfoo/pull/1557

Full Changelog: promptfoo/promptfoo@0.81.4...0.81.5

v0.81.4

Compare Source

What's Changed

Fixes

• fix(redteam): fix modifier handling in PluginBase by @​mldangelo in #​1538
• fix(testCases): improve test case generation with retry logic by @​mldangelo in #​1544
• fix(docker): link peer dependencies in Docker build by @​mldangelo in #​1545
• fix(devcontainer): simplify and standardize development environment by @​mldangelo in #​1547
• fix: redteam progress bar by @​typpo in #​1548
• fix: redteam grading should use defaultTest by @​typpo in #​1549

Refactoring

• refactor: move extractJsonObjects to json utility module by @​mldangelo in #​1539

Chores

• chore(deps): update dependencies by @​mldangelo in #​1540
• chore(deps): bump @​anthropic-ai/sdk from 0.27.0 to 0.27.1 by @​dependabot in #​1541
• chore(deps): bump openai from 4.56.0 to 4.56.1 by @​dependabot in #​1542

Full Changelog: 0.81.3…0.81.4

v0.81.3

Compare Source

What's Changed

• fix(redteam): update logic for json only response format in default provider by @​typpo in https://github.com/promptfoo/promptfoo/pull/1537

Previously

• fix: use redteam provider in extractions by @​typpo in https://github.com/promptfoo/promptfoo/pull/1536
• feat: Indirect prompt injection plugin by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1518
• feat: add support for tags property in config by @​typpo in https://github.com/promptfoo/promptfoo/pull/1526
• feat: ability to reference external files in plugin config by @​typpo in https://github.com/promptfoo/promptfoo/pull/1530
• feat: custom redteam plugins by @​typpo in https://github.com/promptfoo/promptfoo/pull/1529
• fix: remove failure messages from output by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1531
• fix: reduce pii false positives by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1532
• fix: Addtl Pii false positives by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1533
• fix: RBAC plugin false positives by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1534
• fix: redteam providers should be overriddeable by @​typpo in https://github.com/promptfoo/promptfoo/pull/1516
• fix: dont use openai moderation if key not present by @​typpo in https://github.com/promptfoo/promptfoo/pull/1535
  chore: Set jest command line setting for jest extension by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1527

Full Changelog: promptfoo/promptfoo@0.81.0...0.81.3

v0.81.2

Compare Source

What's Changed

• fix: use redteam provider in extractions by @​typpo in https://github.com/promptfoo/promptfoo/pull/1536

Previously

• feat: Indirect prompt injection plugin by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1518
• feat: add support for tags property in config by @​typpo in https://github.com/promptfoo/promptfoo/pull/1526
• feat: ability to reference external files in plugin config by @​typpo in https://github.com/promptfoo/promptfoo/pull/1530
• feat: custom redteam plugins by @​typpo in https://github.com/promptfoo/promptfoo/pull/1529
• fix: remove failure messages from output by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1531
• fix: reduce pii false positives by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1532
• fix: Addtl Pii false positives by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1533
• fix: RBAC plugin false positives by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1534
• fix: redteam providers should be overriddeable by @​typpo in https://github.com/promptfoo/promptfoo/pull/1516
• fix: dont use openai moderation if key not present by @​typpo in https://github.com/promptfoo/promptfoo/pull/1535
  chore: Set jest command line setting for jest extension by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1527

Full Changelog: promptfoo/promptfoo@0.81.0...0.81.2

v0.81.1

Compare Source

What's Changed

• feat: Indirect prompt injection plugin by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1518
• feat: add support for tags property in config by @​typpo in https://github.com/promptfoo/promptfoo/pull/1526
• feat: ability to reference external files in plugin config by @​typpo in https://github.com/promptfoo/promptfoo/pull/1530
• feat: custom redteam plugins by @​typpo in https://github.com/promptfoo/promptfoo/pull/1529
• fix: remove failure messages from output by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1531
• fix: reduce pii false positives by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1532
• fix: Addtl Pii false positives by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1533
• fix: RBAC plugin false positives by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1534
• fix: redteam providers should be overriddeable by @​typpo in https://github.com/promptfoo/promptfoo/pull/1516
• fix: dont use openai moderation if key not present by @​typpo in https://github.com/promptfoo/promptfoo/pull/1535
• chore: Set jest command line setting for jest extension by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1527

Full Changelog: promptfoo/promptfoo@0.81.0...0.81.1

v0.81.0

Compare Source

What's Changed

• feat(report): performance by strategy by @​typpo in https://github.com/promptfoo/promptfoo/pull/1524
• feat: report view sidebar for previewing test failures by @​typpo in https://github.com/promptfoo/promptfoo/pull/1522
• feat(ai21): Add AI21 Labs provider by @​streichsbaer in https://github.com/promptfoo/promptfoo/pull/1514
• feat(docker): add Python runtime to final image by @​mldangelo in https://github.com/promptfoo/promptfoo/pull/1519
• feat(anthropic): add support for create message headers (prompt caching) by @​typpo in https://github.com/promptfoo/promptfoo/pull/1503
• fix(scriptCompletionProvider): handle UTF-8 encoding in script output by @​mldangelo in https://github.com/promptfoo/promptfoo/pull/1515
• fix(config): support loading scenarios and tests from external files by @​typpo in https://github.com/promptfoo/promptfoo/pull/331
• chore: add plugin/strategy descriptions by @​typpo in https://github.com/promptfoo/promptfoo/pull/1520
• chore: add promptfoo redteam plugins command to list plugins by @​typpo in https://github.com/promptfoo/promptfoo/pull/1523
• chore(deps-dev): bump @​aws-sdk/client-bedrock-runtime from 3.635.0 to 3.637.0 by @​dependabot in https://github.com/promptfoo/promptfoo/pull/1513
• chore: clear cache status messages by @​typpo in https://github.com/promptfoo/promptfoo/pull/1517

Full Changelog: promptfoo/promptfoo@0.80.3...0.81.0

v0.80.3

Compare Source

What's Changed

Features

• Add Support for Embeddings API (Cohere): Added support for the embeddings API. #​1502 by @​typpo
• Improve Download Menu: Enhanced the web UI by improving the download menu, adding an option to download human eval test cases, and adding tests. #​1500 by @​mldangelo

Fixes

• Python IPC Encoding: Resolved an issue by ensuring that Python IPC uses UTF-8 encoding. #​1511 by @​typpo

Chores

• Dependencies:
  • Bumped @anthropic-ai/sdk from 0.26.1 to 0.27.0. #​1507 by @​dependabot
  • Upgraded Docusaurus to version 3.5.2. #​1512 by @​mldangelo

---

Full Changelog: 0.80.2...0.80.3

v0.80.2

Compare Source

What's Changed

• fix: remove prompt-extraction from base plugins by @​typpo in https://github.com/promptfoo/promptfoo/pull/1505

Full Changelog: promptfoo/promptfoo@0.80.1...0.80.2

v0.80.1

Compare Source

What's Changed

• refactor(harmful): improve test generation and deduplication by @​mldangelo in https://github.com/promptfoo/promptfoo/pull/1480
• fix: hosted load shared eval by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1482
• fix: Generate correct url for hosted shared evals by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1484
• fix(assertions): update error messages for context-relevance and context-faithfulness by @​mldangelo in https://github.com/promptfoo/promptfoo/pull/1485
• feat: multilingual strategy by @​typpo in https://github.com/promptfoo/promptfoo/pull/1483
• chore(eslint): add and configure eslint-plugin-unicorn by @​mldangelo in https://github.com/promptfoo/promptfoo/pull/1489
• chore(deps-dev): bump @​aws-sdk/client-bedrock-runtime from 3.632.0 to 3.635.0 by @​dependabot in https://github.com/promptfoo/promptfoo/pull/1490
• fix: include vars in python provider cache key by @​typpo in https://github.com/promptfoo/promptfoo/pull/1493
• fix: Including prompt extraction broke redteam generation by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1494
• fix: floating point comparisons in matchers by @​typpo in https://github.com/promptfoo/promptfoo/pull/1486
• feat(redteam): improve test generation and reporting by @​mldangelo in https://github.com/promptfoo/promptfoo/pull/1481
• feat(eval)!: remove interactive providers option by @​mldangelo in https://github.com/promptfoo/promptfoo/pull/1487
• site: enterprise breakdown by @​typpo in https://github.com/promptfoo/promptfoo/pull/1495
• fix: Prompt setup during redteam generation by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1496
• fix: hardcoded injectVars in harmful plugin by @​sklein12 in https://github.com/promptfoo/promptfoo/pull/1498
• site: enterprise blog post by [@​typpo](