StartupSan Francisco, CA · Est. 2023
Braintrust
Enterprise-grade evals, data and AI gateway.
Braintrust turns prompts and agent traces into testable datasets and CI, plus an inference gateway — helping teams measure whether harness changes actually improve agent quality.
Pros & cons
Each point cites its sources and says what kind of evidence backs it. A second reviewer tried to refute every claim before it was kept.
Pros
- User reportsUsers find it easy to set up datasets and run evals across models; one calls it a game changer for checking prompts, especially for classification.[1][2]
- Independent testTracing, evals, datasets and prompts in one product, which a six-month hands-on review called the most complete eval toolchain in 2026.[1]
- Independent testYou can turn any production trace into a golden eval example from the UI, and the free tier includes 1M trace spans and 10K scores a month.[1]
- User reportsPractitioners use it for model comparisons such as long-context retrieval under production constraints.[1]
Cons
- Independent testClosed source with no self-hosting on standard plans; only Enterprise gets a VPC option.[1][2]
- Independent testLock-in risk: datasets, eval configs and trace history live in Braintrust, and the review found the export tooling is not prominent.[1]
- Independent testLLM-as-judge scorers add model costs on top of the platform fee, estimated at $500-2,000/month for 10K evals a day with expensive judges.[1]
- User reportsPro costs $249/month, and one happy user said the core features are simple enough that they would build them themselves next time.[1][2]
- Best for
- Teams that want a polished, hosted eval and tracing workflow from day one.
- Watch out for
- Plan how you will export datasets, and budget for LLM-judge costs.
Assessed Oct 29, 2026 · adversarially reviewedSuggest a correction
People
Founder & CEO
Evals & benchmarks
Open roles (32)
- Commercial Account Executive (West Region)
San Francisco · Full-time · Posted Nov 9, 2026
Go-to-market - Head of GTM Ops
San Francisco · Full-time · Posted Nov 9, 2026
Go-to-market - Solutions Engineer (Central Region)
Austin · Full-time · Posted Nov 9, 2026
Go-to-marketRemote - GTM Engineer
San Francisco · Full-time · Posted Nov 6, 2026
Go-to-market - Growth Marketing Lead
San Francisco · Full-time · Posted Nov 3, 2026
Go-to-market
On X
No posts from them in our X feed yet.
In the news
Community · OpenAI Blog · May 29, 2026
How Braintrust engineers use Codex with GPT-5.5 to run experiments and code faster.
Neighbours in the directory
San Francisco, CA · Est. 2022
The de-facto standard toolkit for building LLM applications and agents.
LangChain makes the LangChain framework and LangGraph, the low-level orchestration standard for stateful, controllable agents, plus the LangSmith platform for tracing, evaluating and monitoring them in production.
~$75M raised
Berkeley, CA · Est. 2020
Observability and evaluation for AI — makers of OSS Phoenix.
Arize traces and evaluates LLM and agent systems in production; their open-source Phoenix tracer is a common instrumentation choice for agent harnesses that need to be debuggable.
San Francisco, CA · Est. 2021
Evaluation platform for LLMs and agentic systems.
Galileo (makers of the Luna evaluation models) scores agent runs for hallucination, tool-use errors and task completion — automated quality gates for harness engineering. Cisco acquired Galileo in April 2026 and folded it into Splunk's agent observability; the old rungalileo.io domain has lapsed.
