Harness Hub
All open source

promptfoo

Test your prompts, agents and RAG — plus red-teaming.

promptfoo/promptfoo

Maintained by promptfoo

promptfoo is an open-source testing framework for LLM apps: eval matrices, assertions, RAG metrics and an AI red-teaming scanner for vulnerabilities. Config-in-YAML, CI-friendly.

What the community says

User reports from Hacker News, Reddit, GitHub and reviews, as of Oct 29, 2026. Not our verdict.

Pros people report

  • Users find it very useful for managing large, interconnected prompt collections and workflow-specific testsHacker News 1Hacker News 2
  • Long-time users praise it and call it an industry standard; open source (MIT), continuing after the OpenAI acquisitionHacker News 1Hacker News 2Hacker News 3
  • Flexible: custom asserts and any OpenAI-compatible or local endpoint (e.g. llama-server)Hacker News 1Hacker News 2
  • Good fit for building your own benchmarks for your workflow, as opposed to generic leaderboardsHacker News

Cons people report

Best for
Config-driven prompt/model regression tests and red-teaming in CI.
Watch out for
Long-term vendor neutrality after the OpenAI acquisition, and dependency weight.

Adopting an open-source harness? Read the best-practices guide.

Activity

Last push
· today
Latest release
code-scan-action-0.2.3
· 3d ago
Last 90 days
13 releases
  1. code-scan-action-0.2.3code-scan-action: 0.2.3

    · 3d ago

    0.2.3 (2026-11-18) Changes Full-repository scans use the patched filesystem MCP server 2.3.0 and can share eligible committed text files from the compared revision with the hosted scanner. Working-tree changes and untracked files stay outside that snapshot. Set diffsOnly: true…

  2. 0.126.0

    · 3d ago

    0.126.0 (2026-11-18) Before upgrading Derived metrics. Check string formulas in derivedMetrics. Arithmetic, comparisons, ternaries and score names such as count still work. Function calls, property access, assignments and formulas longer than 1,024 characters now stop the…

  3. 0.125.1

    · 3w ago

    0.125.1 (2026-10-25) Bug Fixes restore the authentic Discord invite (#11967) (98000c4)

  4. 0.125.0

    · 4w ago

    0.125.0 (2026-10-23) ⚠ BREAKING CHANGES deps: make jks-js an optional peer dependency (#11739) providers: make Claude Agent and Codex SDKs opt-in (#11134) Features add provider-only environment overrides (#11889) (2eed6e1) assertions: add rouge-l and rouge-s assertion types…

  5. code-scan-action-0.2.2code-scan-action: 0.2.2

    · 6w ago

    0.2.2 (2026-10-08) Bug Fixes code-scan: mint OIDC after CLI installation (#11467) (9a404c8)

Checked against GitHub on Nov 21, 2026. Release notes are shortened to plain text.

In research (1)

ECC

affaan-m/ECC

Agent harness performance optimization system: skills, instincts, memory and security for coding agents.

Maintained by Affaan Mustafa

4 pros · 4 cons

cloudflare/security-audit-skill

Cloudflare's skill for multi-phase security audits with independently verified findings.

Maintained by cloudflare

4 pros · 5 cons

alibaba/open-code-review

Alibaba's hybrid deterministic-plus-agent code review CLI.

Maintained by Alibaba

v1.13.13released
4 pros · 4 cons

Connections

Relations from the Harness Hub knowledge graph, as charted in the State of Harness reports.

Maintained by · 1

  • promptfooOrganisation

Similar entries · 2

The Harness Hub newsletter

New harness companies, projects, releases and the monthly State of Harness report, in your inbox.