Opik
comet-ml/opik
Debug, evaluate and monitor agentic workflows.
Opik is Comet's open-source LLM evaluation platform — tracing, prompt playgrounds, dataset-driven evals and online monitoring for RAG and agent systems. The MLflow lineage applied to the agent loop.
In the news
ClickHouse acquires Langfuse
ClickHouse acquires the open-source LLM observability platform Langfuse as part of its $400M Series D at a $15B valuation — the first big consolidation in agent observability. The open-source core remains maintained; the deal signals that agent tracing is becoming a database-scale problem.
DeepSeek ships Harness Desktop; the 'everything is a plugin' harness passes 240k stars
DeepSeek releases a macOS and Windows Desktop app for DeepSeek Harness, its open-source agent harness where everything — tools, sub-agents, even the loop — is a plugin. The repo passed 240k stars within three months of its August launch, making it the fastest-growing harness in the field's history.
HarnessTax: harness choice swings agent cost more than model choice
Arena.ai's HarnessTax study tests 21 model–harness combinations across Claude Code, Codex CLI and Pi, finding harness choice can significantly change cost even at similar task success rates. Weeks earlier, SWE-bench Pro analysis showed swapping harnesses moved pass@1 more than model upgrades (23%→52% on GLM-5.2) — and that harness rankings barely transfer across models.
xAI open-sources Grok Build, its coding agent harness
xAI open-sources Grok Build — a Rust coding agent harness and fullscreen TUI around Grok models — unusually early for a first-party tool. It joins Claude Code, Codex and Gemini CLI as the fourth major first-party harness, now with a fourth open one.
Related projects
DSPy
stanfordnlp/dspy
Programming, not prompting: optimize the whole pipeline.
DSPy introduced programming models for LLM pipelines — declarative modules whose prompts and weights are optimized against metrics. Its optimizer-first philosophy reshaped how production harnesses treat prompts.
Langfuse
langfuse/langfuse
Open-source LLM engineering platform: traces, evals, prompts.
Langfuse is the open-source observability layer for LLM apps — tracing, prompt management, datasets, evals and cost tracking — deployable self-hosted or as a cloud.
Mastra
mastra-ai/mastra
The TypeScript framework for production agents.
Mastra is a full TS-native agent framework — workflows, tool calling, RAG, evals and a dev playground — that has become the default choice for JavaScript teams building agent backends.