StartupSan Francisco, CA · Est. 2024
Patronus AI
Automated evaluation and guardrails for LLM systems.
Patronus AI scores outputs against policy and business criteria in CI and production, catching regressions before users do — 'unit tests for AI'.
Pros & cons
Each point cites its sources and says what kind of evidence backs it. A second reviewer tried to refute every claim before it was kept.
Pros
- MetadataIts open-weights Lynx hallucination judge reports 87.4% on HaluBench, above GPT-4o's 86.5%, and can be self-hosted.[1]
- DocsPercival, an agent eval copilot, detects 20+ failure modes in agent traces and pinpoints the failing step.[1]
- MetadataRaised a $50M Series B in June 2026 ($70M total) and reports more than 15x revenue growth, a figure that has not been independently verified.[1]
- DocsPublishes research benchmarks such as TRAIL and MEMTRACK for agent failures and memory.[1]
Cons
- MetadataLynx is licensed CC-BY-NC-4.0 (non-commercial) and limited to 8k-token sequences.[1]
- Independent testThere is no official public price list; aggregators say enterprise pricing requires contacting sales and Percival is limited to higher tiers.[1]
- DocsThe company now focuses on Digital World Models for frontier labs, and its homepage no longer names its eval products.[1][2]
- Best for
- Teams that need research-grade hallucination and agent-failure evaluation, or labs that need agent simulation environments.
- Watch out for
- Confirm commercial terms for Lynx, and check whether the eval products are still a priority.
Assessed Oct 29, 2026 · adversarially reviewedSuggest a correction
People
Co-founder & CEO
Evals & benchmarks
Co-founder & CTO
Evals & benchmarks
Open roles (14)
- Expert: Interactive Coding, Embedded Firmware (Contract, Remote)
Remote - US · Posted Nov 21, 2026
EngineeringRemote - Expert Support Manager (Contract, San Francisco)
San Francisco · Posted Nov 10, 2026
Operations - Growth Associate (Contract, Remote)
Remote - US · Posted Nov 6, 2026
Go-to-marketRemote - Expert: Interactive Coding, Payments Processing (Contract, Remote)
Remote - US · Posted Nov 3, 2026
EngineeringRemote - Expert: Interactive Coding, Site Reliability Engineering (Contract, Remote)
Remote - US · Posted Nov 3, 2026
EngineeringRemote
On X
No posts from them in our X feed yet.
Neighbours in the directory
San Francisco, CA · Est. 2022
The de-facto standard toolkit for building LLM applications and agents.
LangChain makes the LangChain framework and LangGraph, the low-level orchestration standard for stateful, controllable agents, plus the LangSmith platform for tracing, evaluating and monitoring them in production.
~$75M raised
San Francisco, CA · Est. 2023
Enterprise-grade evals, data and AI gateway.
Braintrust turns prompts and agent traces into testable datasets and CI, plus an inference gateway — helping teams measure whether harness changes actually improve agent quality.
Berkeley, CA · Est. 2020
Observability and evaluation for AI — makers of OSS Phoenix.
Arize traces and evaluates LLM and agent systems in production; their open-source Phoenix tracer is a common instrumentation choice for agent harnesses that need to be debuggable.
Connections
Relations from the Harness Hub knowledge graph, as charted in the State of Harness reports.