StartupSan Francisco, CA · Est. 2023
AgentOps
Agent observability, testing and session replay.
AgentOps instruments agents to record every step, cost and tool call — then lets teams replay sessions and build regression tests from real failures.
Pros & cons
Each point cites its sources and says what kind of evidence backs it. A second reviewer tried to refute every claim before it was kept.
Pros
- MetadataThe SDK and the full app (dashboard plus API backend) are MIT-licensed and self-hostable, with about 5.9k GitHub stars.[1]
- DocsShips integrations for OpenAI Agents SDK, CrewAI, AG2/AutoGen, LangChain, LlamaIndex, Agno and major model providers.[1]
- MetadataHas a free Basic tier and a Pro plan listed at about $40/month.[1]
Cons
- MetadataDevelopment appears to have slowed: the last commit to the main repo was on June 25, 2026, about four months before this check.[1]
- Independent testEvent-based pricing can grow quickly for long-running or looping agents, so high-volume multi-agent workloads need volume modeling first.[1]
- Independent testComparison reviews note it has no built-in evaluation or dataset management, so it covers tracing and monitoring rather than full evals.[1]
- MetadataG2 says it does not have enough reviews to offer buying insight, so there is little independent user evidence.[1]
- Best for
- Python agent builders who want session replays and cost tracking across several agent frameworks.
- Watch out for
- Check recent commit activity and model event volume for looping agents before you pick a paid tier.
Assessed Oct 29, 2026 · adversarially reviewedSuggest a correction
People
Co-founder & CEO
Evals & benchmarks
Co-founder & COO
Evals & benchmarks
Their open-source work
Python · ≈6.0k stars · MIT
AgentOps-AI/agentops
Agent monitoring, session replays and failure analytics.
Maintained by AgentOps
Activity
- Last push
- · 4mo ago
- Latest release
- 0.4.21
· 1y ago - Last 90 days
- 0 releases
- · 1y ago
What's Changed add back lib by @areibman in https://github.com/AgentOps-AI/agentops/pull/1222 updated gitignore by @areibman in https://github.com/AgentOps-AI/agentops/pull/1223 new screenshots by @areibman in https://github.com/AgentOps-AI/agentops/pull/1224 feat(sync): sync…
- · 1y ago
What's Changed Add video embeds to docs by @srilaasya in https://github.com/AgentOps-AI/agentops/pull/1189 AgentOps OSS release by @areibman in https://github.com/AgentOps-AI/agentops/pull/1190 Update backend run instructions in docs by @areibman in…
- · 1y ago
What's Changed docs: update examples from notebooks by @github-actions[bot] in https://github.com/AgentOps-AI/agentops/pull/1154 Create openai responses api test by @areibman in https://github.com/AgentOps-AI/agentops/pull/1156 Migrate from pyautogen to ag2 Library by…
- · 1y ago
What's Changed [tests] coverage dopamine by @bboynton97 in https://github.com/AgentOps-AI/agentops/pull/1126 fix mcp docs by @michi-okahata in https://github.com/AgentOps-AI/agentops/pull/1129 Cursor/improve agno tool integrations py output d88d by @areibman in…
- · 1y ago
What's Changed Add separate docs for OpenAI Agents SDKs by @areibman in https://github.com/AgentOps-AI/agentops/pull/1080 Update session management documentation to reflect current trace-based architecture by @devin-ai-integration in…
Checked against GitHub on Nov 21, 2026. Release notes are shortened to plain text.
On X
No posts from them in our X feed yet.
Neighbours in the directory
San Francisco, CA · Est. 2022
The de-facto standard toolkit for building LLM applications and agents.
LangChain makes the LangChain framework and LangGraph, the low-level orchestration standard for stateful, controllable agents, plus the LangSmith platform for tracing, evaluating and monitoring them in production.
~$75M raised
San Francisco, CA · Est. 2023
Enterprise-grade evals, data and AI gateway.
Braintrust turns prompts and agent traces into testable datasets and CI, plus an inference gateway — helping teams measure whether harness changes actually improve agent quality.
Berkeley, CA · Est. 2020
Observability and evaluation for AI — makers of OSS Phoenix.
Arize traces and evaluates LLM and agent systems in production; their open-source Phoenix tracer is a common instrumentation choice for agent harnesses that need to be debuggable.
Connections
Relations from the Harness Hub knowledge graph, as charted in the State of Harness reports.