Inspect
UKGovernmentBEIS/inspect_ai
The evaluation framework used by a national safety institute.
Inspect is the UK AI Safety Institute's open-source framework for large-scale model evaluations — tasks, solvers, scorers and tool sandboxes. The serious-science end of agent evals.
In the news
HarnessTax: harness choice swings agent cost more than model choice
Arena.ai's HarnessTax study tests 21 model–harness combinations across Claude Code, Codex CLI and Pi, finding harness choice can significantly change cost even at similar task success rates. Weeks earlier, SWE-bench Pro analysis showed swapping harnesses moved pass@1 more than model upgrades (23%→52% on GLM-5.2) — and that harness rankings barely transfer across models.
Runta raises $20M seed to 'parent' AI agents
a16z leads a $20M seed into Runta at a $100M+ valuation for its agent execution layer — sandboxes, spending caps and audit trails that treat agents like children who need supervision. Martin Casado called it 'core systems software rebuilt for agents'; the round came after prompt-injection containment failures made agent governance urgent.
ClickHouse acquires Langfuse
ClickHouse acquires the open-source LLM observability platform Langfuse as part of its $400M Series D at a $15B valuation — the first big consolidation in agent observability. The open-source core remains maintained; the deal signals that agent tracing is becoming a database-scale problem.
Related projects
DSPy
stanfordnlp/dspy
Programming, not prompting: optimize the whole pipeline.
DSPy introduced programming models for LLM pipelines — declarative modules whose prompts and weights are optimized against metrics. Its optimizer-first philosophy reshaped how production harnesses treat prompts.
Langfuse
langfuse/langfuse
Open-source LLM engineering platform: traces, evals, prompts.
Langfuse is the open-source observability layer for LLM apps — tracing, prompt management, datasets, evals and cost tracking — deployable self-hosted or as a cloud.
Mastra
mastra-ai/mastra
The TypeScript framework for production agents.
Mastra is a full TS-native agent framework — workflows, tool calling, RAG, evals and a dev playground — that has become the default choice for JavaScript teams building agent backends.