CommunityArena.ai · Sep 16, 2026
HarnessTax: harness choice swings agent cost more than model choice
Arena.ai's HarnessTax study tests 21 model–harness combinations across Claude Code, Codex CLI and Pi, finding harness choice can significantly change cost even at similar task success rates. Weeks earlier, SWE-bench Pro analysis showed swapping harnesses moved pass@1 more than model upgrades (23%→52% on GLM-5.2) — and that harness rankings barely transfer across models.
Filed underevalsbenchmarksharnesscostresearch
Related stories
Open Source · Hacker News · Nov 20, 2026
Hi HN! Utpal here, I'm one of the founders of OpenComputer. OpenComputer is an open source cloud coding agent for teams that can run in your own cloud account (AWS, GCP, Azure). We run the agent harness and the sandbox on the same durable VM (architecture in the docs [1]). Fir...
Community · GitHub Changelog · Nov 17, 2026
Today, we’re expanding GitHub Copilot Fusion with new models, available on more Copilot surfaces, and results on two public Terminal-Bench leaderboards, giving you more ways to use Fusion and compare… The post GitHub Copilot Fusion updates: more availability, benchmarks, and m...
Community · GitHub Changelog · Nov 17, 2026
You can now use GitHub Copilot Evals to optimize GitHub Copilot for yourself. As your prompts, models, skills, and tools change, are you still getting the most out of Copilot?… The post GitHub Copilot Evals is now in public preview appeared first on The GitHub Blog .
Community · The Decoder · Nov 17, 2026
OpenAI releases GPT-6.2 Astra and Sol, two new models that are more capable and cheaper than their predecessors. In the independent Artificial Analysis Intelligence Index, Astra ranks second behind Claude Fable 5.2, and it is the first released model to reach the High threshol...