Open sourcePython · ≈1.1k stars · Apache-2.0
Terminal-Bench
Hard, containerized terminal tasks, scored end to end.
Terminal-Bench is the terminal-task benchmark coding agents now cite next to SWE-bench: hard, containerized tasks scored end to end. Version 2.0 runs on the Harbor evaluation framework; the 1.0 task set lives on in the org's repos.
Activity
- Last push
- · yesterday
- Latest release
- v4.1.0
· 2w ago - Last 90 days
- 2 releases
- · 2w ago
Published dataset: https://hub.harborframework.com/datasets/terminal-bench/terminal-bench/5 Terminal-Bench 4.1 implements verifier fixes, reproducibility improvements, hardening against reward hacking. Also implements a new CPU-only dataset containing 63 tasks. Tasks added or…
- · 2mo ago
Release blog: https://www.tbench.ai/news/terminal-bench-4-0 Published dataset: https://hub.harborframework.com/datasets/terminal-bench/terminal-bench/4 Tasks removed: cli-2ph-simplex erp-procurement-planning exam-pdf-eval fix-uautomizer-soundness gpt2-codegolf ico-path-patch…
- · 4mo ago
Added initial 74 Tasks atrx-vep-crispr — @scaleai-bot batched-eval-parity — @tommasocerruti biped-contact-dynamics — @QingquanBao bun-sourcemap-leak — @sherifarm cad-model — @k-rl cargo-flight-dispatch — @pankajjajoo-turing cli-2ph-simplex — @sherifarm coq-block-bound —…
Checked against GitHub on Nov 21, 2026. Release notes are shortened to plain text.
In the news
Community · GitHub Changelog · Nov 17, 2026
Today, we’re expanding GitHub Copilot Fusion with new models, available on more Copilot surfaces, and results on two public Terminal-Bench leaderboards, giving you more ways to use Fusion and compare… The post GitHub Copilot Fusion updates: more availability, benchmarks, and m...
In research (13)
- Higher-Scoring Coding Models Did Not Get More Work DoneVerdict: Useful
Nov 17, 2026 · Discussed as a benchmark where Fable 5 led Opus 4.8 by only 1.6 points, which the paper attributes largely to safety-refusal fallbacks.
- From Repositories to Training Environments: Scaling Long-Horizon Coding Agents with Environment Evolution LoopVerdict: Useful
Nov 16, 2026 · Terminal-Bench 2.1 is one of the five evaluation benchmarks.
Nov 16, 2026 · Cited via Vals AI's Terminal-Bench 2.1 reporting as a comparison for handling safety-system fallbacks.
Oct 27, 2026 · Terminal-Bench Lite is the coding-harness evaluation.
- AutoRSE: Evaluating LLM Agents as Recommendation Engineers for Self-Evolving Recommender SystemsVerdict: Useful
Oct 27, 2026 · Terminus-2 is the Terminal-Bench reference agent, used here as a harness.
- Work Agents: Systems, Capabilities, and OptimizationVerdict: Incremental
Oct 26, 2026 · Representative terminal-task benchmark in Tab. 12.
Showing the 6 newest of 13 papers.
Neighbours on the workbench
Go · ≈55k stars · Apache-2.0
alibaba/open-code-review
Alibaba's hybrid deterministic-plus-agent code review CLI.
Maintained by Alibaba
v1.13.13releasedPython · ≈39k stars · MIT
stanfordnlp/dspy
Programming, not prompting: optimize the whole pipeline.
Maintained by Stanford NLP
3.5.0releasedTypeScript · ≈38k stars · Other
langfuse/langfuse
Open-source LLM engineering platform: traces, evals, prompts.
Maintained by Langfuse
v4.83.0releasedConnections
Relations from the Harness Hub knowledge graph, as charted in the State of Harness reports.
Maintained by · 1
- HarborOrganisation
Mentioned in · 7
- Beyond Weight Updates: Evolving Agent Harnesses for Domain SpecializationPaper · 2026-11-19
- Clover-35B-A3B-Preview Technical ReportPaper · 2026-11-18
- GitHub Copilot Fusion updates: more availability, benchmarks, and modelsNews · 2026-11-17
- The Last Mile to the Real World: Agent-Native Primitives as Reusable Building Blocks for Solving Real-World TasksPaper · 2026-11-05
- Your Policy Is Secretly a Harness: The Decision Geometry of Agent InterfacesPaper · 2026-10-27
- CTXAgent - Self-Controlled Context Management of LLM AgentsPaper · 2026-10-25
and 1 more
