Harness Hub
All papers

Identifying and Repairing Energy Smells with Agent Skills

Mohammadjavad Mehditabar, Saurabhsingh Rajput, Tushar Sharma

Verdict: Incremental

The skill design is a reasonable, reusable pattern: staged disclosure, stage-gated tools, an external input generator and verifier, and post-hoc re-measurement. The evaluation, however, is single-run, uses three repos, and has no external baseline. Behavior preservation rests on a handful of generated inputs. The abstract's main comparative claim is contradicted by Table I for verified saving repairs.

  • Novelty

    3 out of 5
  • Rigor

    2 out of 5
  • Reproducibility

    2 out of 5
  • Relevance to harnesses

    3 out of 5

TL;DR

Dr. Greens is a Claude Code agent skill that detects and repairs energy smells in Python. It loads a 65-root-cause taxonomy, a static analyzer, and an external verifier that measures energy. Across 18 single-run sessions on 3 repositories, 68.6% of 258 measured repairs saved energy, with a median saving of 24.9%. The abstract claims the full skill found more smells with verified savings than the no-taxonomy baseline in every repo/model pair. The paper's own Table I contradicts that claim in two of six pairs.

Adversarial review by Harness Hub · Nov 20, 2026 · read full text · Suggest a correction

Original abstract (authors’ text, select to expand)

Energy smells are inefficient implementations that waste energy. Once code reaches production, these smells become costlier and riskier to fix, so energy efficiency checks should shift left, i.e., move earlier in the development lifecycle, to where code is written. Existing energy smell tools rely on fixed static rules and do not measure energy savings, whereas LLM-based optimizers mostly target execution time and rarely ground their changes in concrete smell definitions. To address both limitations, we present DR. GREENS, an agent skill that detects and repairs energy smells in Python code within the developer's coding environment. The skill grounds the coding agent in our taxonomy of 12 energy smells and 65 root causes, augmented with repair guidance drawn from 230 measured cases. The skill also encapsulates a static analyzer and an independent verifier to confirm behavior preservation and energy saving for each attempted repair. We evaluated DR. GREENS in 18 sessions on three open-source repositories, with two models and three skill configurations. Of the 258 measured repairs, 68.6% saved energy, with a median saving of 24.9%. Compared with a baseline skill that lacks the taxonomy, the full skill found more energy smells with verified savings in all three repositories with both models. DR. GREENS helps developers detect energy hotspots and fix them reliably without altering program behavior, which makes energy-efficient development more feasible.

Filed undercoding-agentsagentsllm

Summary

The paper packages the authors' energy-smell taxonomy as an agent skill for Claude Code. The taxonomy has 12 smells and 65 root causes. It is delivered in three levels of progressive disclosure: an index, 12 category cards, and 65 root-cause cards. The cards draw on 230 author-written before/after cases with measured results. The skill also bundles four tools: - a static analyzer with 40 custom AST rules plus 31 Ruff rules, covering 49 of 65 root causes; - an energy probe; - an LLM input generator; - a verifier. The agent works in four steps: 1. Find: favors recall, can be split across subagents, and is cross-checked against the analyzer. 2. Decide: favors precision, using the root-cause cards and their near-misses. 3. Repair and Measure: whole-function replacements are submitted to the verifier. 4. Finish. The input generator is a separate model, DeepSeek V4.1 Flash. Inputs are filtered following GSO and COFFE. The verifier compares return values, argument state, printed output and written files. It then measures energy with CodeGreen using a paired protocol: nine pairs, a Student's t confidence interval, and a noise floor. The skill limits how many repairs a session can measure. The paper does not state the limit, but Table I denominators never exceed 20. The evaluation runs three skill variants (Core, Core+Tax, Full) with Claude Sonnet 5.5 and Claude Haiku 5.5 on three repositories. These were the ones out of the first ten SEART results that installed and ran. That gives 18 sessions, each run once. Every repair is re-measured after the session.

Key findings

  • Section III: 317 repairs were submitted and 258 received a measured verdict. Of these, 68.6% saved energy (95% CI 63.3% to 73.2%), with a median saving of 24.9% (95% CI 17.8% to 26.7%). Both intervals come from a bootstrap over the 18 sessions.
  • Section III: Sonnet saved energy on 67.3% of its measured repairs and Haiku on 70.7%.
  • Section III: total saving repairs were 48 for Core, 66 for Core+Tax and 63 for Full.
  • Section III: Core had the highest median saving in five of six repository/model combinations. On functions that two skills both repaired with the same model, the median difference in best saving was 0.
  • Section III: with Sonnet, the taxonomy skills repaired 36 functions that Core missed, with a median saving of 8.2%.
  • Section III: after deduplication, 192 distinct edits remain and 65.6% of them save energy. Edits to analyzer-flagged functions save energy more often (69.0% vs 62.0%). Their median saving, however, is 17.5 percentage points lower (95% CI 2.6 to 38.4).
  • Section III, Opus run on sample-apps only: 69 smelly functions with Core, 146 with Core+Tax and 131 with Full.
  • Table I: Full has fewer saving repairs than Core on dx-toolkit/Sonnet (10/20 vs 13/20) and on community/Haiku (3/5 vs 4/6).
  • Section II-C: of the 230 reference cases, 146 save energy on some input and cost more on none. 59 of the 65 root causes have at least one such case.

The paper at a glance

01Step 1 Find: agentreads everyfunction with thetaxonomy index andcategory cards…02Step 2 Decide:root-cause cardsand near-missesbecome available;agent accepts or…03Inputs: DeepSeekV4.1 Flash writesedge inputs and 8candidate loadinputs; keep…04Step 3 Repair:agent submits fullfunctionreplacements;verifier checks…05Verdicts: confirmed/ partial / mixed /regressed /negligible counttoward the session…06Post-sessionre-measurement ofevery repair;recommend the bestconfirmed repair…07Step 4 Finish:summary of repairs,root causes andverdicts
  1. Step 1 Find: agent reads every function with the taxonomy index and category cards (subagents on large repos), then reviews static analyzer findings it missed
  2. Step 2 Decide: root-cause cards and near-misses become available; agent accepts or rejects each suspicion
  3. Inputs: DeepSeek V4.1 Flash writes edge inputs and 8 candidate load inputs; keep deterministic inputs and the 4 highest-energy ones covering at least 40% of statements
  4. Step 3 Repair: agent submits full function replacements; verifier checks equivalence on the generated inputs, then runs paired energy measurement (9 pairs, CI, noise floor)
  5. Verdicts: confirmed / partial / mixed / regressed / negligible count toward the session limit; not-equivalent or failed repairs can be resubmitted; inconclusive when noise is high
  6. Post-session re-measurement of every repair; recommend the best confirmed repair per function
  7. Step 4 Finish: summary of repairs, root causes and verdicts
Fig. 1 — How the paper works, step by step (arXiv:2611.23203).
Core (no taxonomy)48 repairsCore+Tax66 repairsFull (Dr. Greens) (this paper)63 repairs
Fig. 2 — Repairs that saved energy, summed over 3 repos and 2 models, higher is better. Source: Section III, 'Wider search adds smaller savings', arXiv:2611.23203.
Data table
Repairs that saved energy, summed over 3 repos and 2 models
Methodrepairs
Core (no taxonomy)48 repairs
Core+Tax66 repairs
Full (Dr. Greens) (this paper)63 repairs

Why it matters

For harness builders this is a worked example of a domain skill that uses progressive disclosure and stage-gated tools. The index and category cards load in the search step. The detailed cards unlock in the decision step. The verifier and input generator become available only after Step 2. The more transferable idea is keeping evidence out of the agent's control. A separate LLM generates the inputs, an external verifier checks every repair, and every repair is re-measured after the session, so the agent cannot pick the inputs it is graded on. That pattern applies to any skill that asks an agent to optimize a measurable property. The energy-specific content matters only to teams that care about Python energy use. The evaluation is too small to say how much the taxonomy layer adds over a plain verifier loop: one run per cell, three repos, Anthropic models only, no external baseline. The paper's own numbers suggest the knowledge layers mainly add repairs with smaller savings, not larger ones.

Issues (8)

  • High severity

    The abstract claims that, compared with the no-taxonomy baseline, the full skill found more energy smells with verified savings in all three repositories with both models. Table I contradicts this for saving repairs in two of six pairs. On dx-toolkit with Sonnet, Full has 10/20 saving repairs against Core's 13/20. On community with Haiku, Full has 3/5 against Core's 4/6. The claim holds only for the 'Smelly' column, which counts accepted suspicions, not verified savings. Core also has the highest median saving in five of six combinations.

  • High severity

    Behavior preservation is checked only on a few LLM-generated edge inputs plus four load inputs. The comparison covers return value, argument state, printed output and files written. The repositories' own test suites are not run and no human reviews the repairs. The abstract's statement that the tool fixes smells 'reliably without altering program behavior' is not established by this oracle. Inputs are also filtered to those that give the same output twice, so nondeterministic paths are never checked.

  • High severity

    Each of the 18 configurations is run once. The bootstrap resamples the 18 sessions as if they were independent, although they share 3 repositories and often write the same edit. The paper itself deduplicates to 192 distinct edits. Per-cell skill comparisons, such as 'more smelly functions on every repository', are therefore single-sample differences with no variance estimate.

  • Medium severity

    The session limit on measured repairs is not stated numerically, and sessions used very different numbers of measured repairs. For example, Core on dx-toolkit with Haiku measured 1/5, Core on community measured 7/8 and 4/6, and Full on dx-toolkit with Haiku measured 10/12. Totals like 48 vs 66 vs 63 therefore partly reflect how many repairs each session got measured, not detection quality alone. No token, time or cost figures are reported, so effort cannot be normalized.

  • Medium severity

    The 68.6% headline uses a selected denominator. It counts only the 258 of 317 submitted repairs that received a measured verdict. 'Saves energy' also includes 'partial' repairs that save on only some of the four inputs. Savings are measured on function-level microbenchmarks of about three seconds of repeated calls, not whole-program or production workloads.

  • Medium severity

    There is no external baseline, only an internal ablation (Core vs Core+Tax vs Full). The paper names prior tools in its gap analysis but compares against none of them: ecoCode/creedengo, EffiSkill, PerfAgent, LASSI-EE and EnergAgent. A plain Claude Code agent without the skill's verifier loop is also not tested.

  • Medium severity

    External validity is narrow. There are 3 repositories, selected as those of the first ten SEART results that installed and ran. All measurement is on one server (two Intel Xeon E5-2650). Only Anthropic models drive the agent, and Opus was run on one repository. The authors argue that energy savings depend on hardware and workload, yet they measure on a single platform.

  • Low severity

    The reference knowledge is author-written, and only 146 of the 230 cases cleanly save energy. The analyzer is said to be validated on 515 test cases, 319 of them must-not-report. Its precision and recall on those cases are not reported. The replication package is cited, but no URL appears in the text.

Questions for the authors

  • How do you reconcile the abstract's claim that the full skill found more smells with verified savings in every repo/model pair with Table I? Full has fewer saving repairs than Core for dx-toolkit/Sonnet (10 vs 13) and community/Haiku (3 vs 4).
  • Did any recommended repair fail the target repository's own test suite, and was that checked?
  • What is the run-to-run variance of a single configuration repeated several times?
  • What is the per-session limit on measured repairs? Why did some sessions measure far fewer repairs than others?
  • How many tokens, how much wall-clock time and how many subagent calls did each skill variant use per session?
  • How does a plain Claude Code agent with only the energy probe and an instruction to reduce energy compare against Core?
  • What are the precision and recall of the 40 custom analyzer rules on the 515 test cases?
  • Do the measured savings hold on a second hardware platform?

Related in the directory

Open-source projects

All 18 sessions run inside Claude Code with permission to run any command.
Dr. Greens is built as an Anthropic agent skill and follows its progressive-disclosure design.
The paper relies on the agent skill format (instructions, reference files and scripts loaded on demand).

Commercial products

Commercial LLM-based Python optimizer targeting execution time, the class of tool the paper positions against.

Code2Video uses agent-authored 3D proxies to guide video generation, enabling explicit control over structure and dynamics. However, how to construct proxies and prompts for optimal downstream quality remains underexplored, as do the capabilities and limitations of this paradigm. Through controlled experiments and blinded human evaluation, we study proxy granularity, physics simulation, and prompting. The results reveal task-dependent trade-offs: coarse proxies favor natural motion, while medium-grained proxies favor consistency. Physics simulation and proxy-aligned prompting further improve average quality. Guided by these findings, we introduce Coda-DaVinci, a task-adaptive agentic framework for Code2Video generation that selects between proxy-guided and direct generation, adapts modeling granularity to each entity, and coordinates simulation with proxy-grounded prompting. Its multi-agent harness preserves scene assets and guides repairs through visual inspection and independent review. Experiments demonstrate leading instruction following on VTR and scene consistency on R2M, improved physical plausibility over direct generation, and higher visual realism than the compared Code2Video baselines. Long-video evaluations further show gains in event structure and consistency across successive interactions, while revealing limitations in transition stability and adherence to scene settings.

Yiyan Ji, Qixun Wang +11

Agents based on large language models (LLMs) can now delete files and send payments on behalf of their users. News media reports routinely describe failures of these agents (e.g., Replit's coding agent deleting a production database), but a report often only says what went wrong and rarely (in)validates the incident or finds its root cause. Incident catalogs collect these reports, and benchmarks record whether agents fail, but neither replicates an incident nor tests the cause its report might identify. In this paper, we present ROGUE-AI (Replication of Agentic Security Events: Audit and Investigation), a multi-agent framework that closes the above gap and rebuilds each reported incident offline together with a matched control that lacks only the blamed cause, and runs agents on both. Our results suggest that ROGUE-AI is able to reproduce the reported harm in 89% of its 108 attempted replications. Agents built on five current GPT, Claude, Gemini and DeepSeek models made the reported mistake in 18% to 33% of runs, against at most 1% without the blamed cause. Of the 72 incidents with real-world harm, 62 had no underlying attacker, and in these, agents did what their task asked for but missed an ordinary hazard, such as a database shared with the live site. Replications without an attacker caused more harm than replications of prompt injection for all five models. Defenses against prompt injection look for injected instructions, but these replications contain none. Further, a warning after each tool output that names the hazard and its consequence lowered harm for all five models, whereas a note of the same length that does not mention the hazard left most of the harm in place. Built from real incidents, the ROGUE-AI replications form a reusable audit framework that vendors can rerun before each new release.

Mengxiao Wang, Mahbub Alam +2

Pretrained language models (LMs) remain unreliable on exact multi-digit arithmetic without gradient fine-tuning or external tool interpreters. We introduce closed-form, zero-gradient circuit-grafting primitives that compile deterministic computation directly into pretrained LM weights using zero training examples and zero runtime hooks, while bounding per-layer normalization drift and preserving natural-language (NL) competence. Our construction combines three analytical mechanisms: (1) two-layer Mellin-GLU quadrature over bias-free SwiGLU/GeGLU layers, turning unknown per-token RMSNorm scaling into translation along a log-spaced gate grid ($>250\times$ dynamic range) with IEEE 754 error bounds; (2) cross-token scale cancellation via signed Chebyshev Indefinite-RoPE self-sink attention and memoryless SiLU-antisymmetric GatedDeltaNet shifts; and (3) axis-aligned width extension ($d \to d+K$) with $\sqrt{d/(d+K)}$ norm compensation, Base-1000 canonical BPE emission, and differential $(+254,-256)$ tied unembeddings. An automated 5-stage compiler synthesizes 4-8 digit signed $\{+,-,\times\}$ circuits in constant $\mathsf{TC}^0$ depth alongside 3-literal Boolean (100%, $N=1{,}728$; plus 42-layer Gemma 4 E4B audits) and 5-bit Majority/Parity (100%) circuits. On 6-digit arithmetic ($N=1{,}200$, BF16), IBM Granite 4.2 3B (1.07M-1.33M active non-zeros, $\le 0.036\%$) and Alibaba Qwen 3.5 4B (2.25M-2.39M non-zeros, $\le 0.057\%$) reach 99.17% and 100.00% ALU-prefix accuracy (100% with an FP32 sidecar) and 98%-100% mid-sentence resumption, outperforming 12.8K-sample LoRA (42.2% / 61.7%, 0% inline continuation). A factorized block-sparse sidecar holds A100 memory at 7.89 / 10.65 GiB (4.60 / 8.14 GiB under INT8/INT4 + FP32 sidecar) while passing all 14/14 pre-registered $>2$K-token Tango splits ($N=4{,}392$; 0/11,134 OOD triggers).

Tanmay Sah

The Harness Hub newsletter

New harness companies, projects, releases and the monthly State of Harness report, in your inbox.