Harness Hub
All papers

Mitigating Catastrophic Forgetting with Physics-Indexed Specialist Memory: the Evolutionary Neuroplastic Agent

Blas Eugenio Vicco Zarate, Miguel Hernán Vicco

Verdict: Incremental

The paper combines known components (context detection, a per-context frozen policy archive, Bayesian policy reuse, triggered hypermutation and random immigrants) in a gradient-free setting, and reports the results candidly. Its own reference baselines beat it on adaptation speed and return. Its ablation confounds interaction budget, archive capacity, resets and test-time selection with the skills under test. Its primary HalfCheetah metric favours weak learners.

  • Novelty

    2 out of 5
  • Rigor

    2 out of 5
  • Reproducibility

    3 out of 5
  • Relevance to harnesses

    1 out of 5

TL;DR

ENA is a gradient-free neuroevolution agent. It freezes successful small policies into an archive indexed by detected physics regime and recalls them with an adapted Bayesian policy reuse. It reports 83.2% mean test success over five non-stationary CartPole environments in a 30-replicate ablation. By its own account it adapts more slowly than PPO on CartPole and earns lower returns than SAC on HalfCheetah. This is continual control RL with essentially no bearing on agent harnesses.

Adversarial review by Harness Hub · Nov 18, 2026 · read full text (truncated) · Suggest a correction

Original abstract (authors’ text, select to expand)

Continual reinforcement learning requires adaptation to changing conditions while preserving performance under previously encountered dynamics. It was investigated whether specialists generated by evolutionary search can provide a reusable memory when combined with environment detection and failure-driven search. The Evolutionary Neuroplastic Agent (ENA) evolves small neural networks without gradients, archives selected policies by detected dynamic regime, and retrieves them through Bayesian policy reuse. The architecture was evaluated in non-stationary CartPole and HalfCheetah environments. In the 30-replicate CartPole ablation, the complete architecture achieved a mean success rate of 83.2% across the five test environments. Among variants that learned the harder CartPole environments, the complete architecture showed the most consistent retention. A selection ablation also showed that the retrieval rule affected performance. In the evaluated sequences, ENA adapted more slowly than PPO on CartPole and achieved lower returns than SAC on HalfCheetah. The evidence concerns the complete architecture and does not isolate gradient-free optimization as the cause of retention or establish superiority over other continual-learning methods.

Authors’ note

25 pages, 7 figures, 23 tables. Code and data: https://github.com/blasvicco/ena/tree/v1

Filed undermemoryagents

Summary

ENA keeps a population of tiny MLP policies (4x2 ReLU on CartPole, 32x16 tanh on HalfCheetah) and evolves them by mutation, crossover and fresh immigrants. Each policy carries a Trust score. A failure-driven Plasticity value sets three things: how many candidates are re-evaluated, for how many steps, and the mutation noise. Policies that pass a z-score or absolute Trust gate are frozen into a Hall of Fame, partitioned into niches. Niches come from different detectors in the two domains. On CartPole they come from linear-Gaussian dynamics models with a Predictive-Mixture CuSum detector. On HalfCheetah, where those models create far more niches than there are environments, they come from a fixed 64-step wavelet chirp played after an environment reset. At test, weights are frozen. A modified BPR, with a performance model learned online and a Fixed-Share belief over niches, picks the acting archived policy. It combines return and dynamics or chirp likelihood. The studies are: - a 30-replicate CartPole skill ablation (3 training, 5 test environments), budgeted in episodes; - a selection-rule ablation; - an equal-environment-step reference comparison against custom DQN, PPO and SAC, with 50 replicates on CartPole and 20 on HalfCheetah; - a HalfCheetah detection benchmark; - a Plasticity-controller and compute study. The authors explicitly disclaim superiority over continual-RL methods. They also say the evidence does not isolate gradient-free optimisation as the cause of retention. The results section was truncated in the provided text, so the ablation tables were not seen.

Key findings

  • Abstract: in the 30-replicate CartPole ablation, the complete architecture reached a mean success rate of 83.2% across the five test environments. The per-variant numbers were not visible in the truncated text.
  • Section IV-B2: ENA spends 84.9% of its CartPole training steps evaluating candidates. It plays about 815 episodes per environment, against 6,759 for DQN, 5,791 for PPO and 5,927 for SAC-Discrete, at a budget of 2,500,000 steps per environment.
  • Section IV-B2: on HalfCheetah, ENA plays about 214 episodes per environment, against 1,000 for PPO and 1,000 for SAC, at 1,000,000 steps per environment.
  • Abstract and Section IV-A2: ENA adapted more slowly than PPO on CartPole and had lower returns than SAC on HalfCheetah. ENA never reaches the HalfCheetah threshold of 4,800 at test, so retention replaces reliability as the primary metric there.
  • Section III-H5: on HalfCheetah the transition-model detector confirms many more changes, and creates many more niches, than there are environments, even when a single fixed gait is played. The cause was not isolated. A reset-based 64-step chirp, 6.4% of an episode, replaces it.

The paper at a glance

01Population of smallMLP specialistswith Trust scores02Act one episode(epsilon-greedy onTrust) and feedtransitions to thedetector03Failure signal setsPlasticity: numberand length ofcandidateevaluations, and…04Gate the actingspecialist into theHall of Fame underits detected niche(PM-CuSum on…05Evolve: keepelites; replace therest with mutants,crossovers andfresh specialists06Test: frozenweights; Bayesianpolicy reuse overniches picks thegladiator
  1. Population of small MLP specialists with Trust scores
  2. Act one episode (epsilon-greedy on Trust) and feed transitions to the detector
  3. Failure signal sets Plasticity: number and length of candidate evaluations, and mutation size
  4. Gate the acting specialist into the Hall of Fame under its detected niche (PM-CuSum on CartPole, chirp on HalfCheetah)
  5. Evolve: keep elites; replace the rest with mutants, crossovers and fresh specialists
  6. Test: frozen weights; Bayesian policy reuse over niches picks the gladiator
Fig. 1 — How the paper works, step by step (arXiv:2611.21598).

Why it matters

This paper is essentially irrelevant to people building agent harnesses (coding agents, tool loops, memory layers, eval rigs). It studies tiny control policies on CartPole and HalfCheetah. Only physical parameters change, and the method relies on Gymnasium environment resets for candidate evaluation and probing. The one transferable idea is not new: keep a frozen archive of past solutions keyed by an automatically detected context, and choose among them with a belief that mixes context evidence and observed reward. That resembles routing a task to a stored skill or playbook using a context classifier plus success history. The paper gives no evidence that this works outside low-dimensional RL. Its own HalfCheetah result is the more useful warning: the learned-dynamics context detector fragmented into spurious niches once the signal was noisy and policy-dependent, and had to be replaced by a hand-designed probe. Read it only if you work on continual RL or neuroevolution.

Issues (9)

  • High severity

    The skill ablation, the primary evidence for RQ1, is budgeted in episodes, not environment steps. Variants with Plasticity on evaluate many more candidates and interact far more with the environment. The 83.2% headline for the complete architecture therefore mixes the effect of the skills with extra interaction relative to the Plasticity-off arms.

  • High severity

    Switching environment detection on changes three things at once. It sets the archive index, raises the archive capacity from K in total to K per niche, and adds resets of the return history, input statistics, Trust rate and Plasticity. The paper concedes the ablation does not separate them. RQ1a, whether physics indexing is necessary, is therefore not answered by the ablation.

  • High severity

    The test protocols are asymmetric. Variants without a Hall of Fame play a random member of the final population. Variants with one get Bayesian policy reuse, whose belief and performance model keep adapting at test. In the baseline comparison, DQN, PPO and SAC act deterministically with frozen parameters, while ENA still adapts its belief, its performance model and its detector alarms. Retention gains are partly attributable to test-time selection and adaptation that the comparators lack.

  • High severity

    On HalfCheetah, retention (test return as a percentage of end-of-phase training return) is the primary metric because ENA never reaches the threshold. The paper itself notes that retention can exceed 100% and does not penalise an agent for having learned little. ENA has lower returns than SAC. A ratio metric favours a weak learner, and the absolute-forgetting family for HalfCheetah was defined after the runs.

  • Medium severity

    No continual-RL baseline is run: no EWC, CLEAR, OWL, MBCD, SWOKS, TeLAPA or Nisioti et al. The single-policy baselines are custom implementations with default hyperparameters plus two ad hoc fixes found in pilot runs. ENA's HalfCheetah settings were tuned on separate seeds, so tuning effort is asymmetric. Even so, ENA is slower than PPO on CartPole and worse than SAC on HalfCheetah. The title's claim of mitigating catastrophic forgetting is untested against methods built for it.

  • Medium severity

    The HalfCheetah chirp excitation and features were chosen by the ablation in Section V-F, on the three HalfCheetah training environments. That same detection benchmark is cited as evidence for RQ1c, so selection and evaluation overlap. The CartPole detector thresholds tau and gamma were set on preliminary CartPole runs.

  • Medium severity

    Candidate fitness is extrapolated from truncated rollouts. With Plasticity off on CartPole, surviving 10 steps is credited as the maximum return of 500. On HalfCheetah, returns from at most 200 of 1,000 steps are rescaled, and the authors did not check whether this ranks specialists correctly. The Plasticity-off arm is handicapped by its fitness signal, not only by reduced search.

  • Low severity

    Stored Trust values in the Hall of Fame are computed against a return history that changes and is cleared at each change. They are therefore not on one scale. An early, weaker gladiator can block a stronger later one from a full niche.

  • Low severity

    The reproducibility claim ('released code and experiment outputs') is not backed by any URL in the provided text. The truncated results section also prevented checking the per-variant numbers behind the claim of 'most consistent retention'.

Questions for the authors

  • What are the skill-ablation reliability and retention when every variant gets the same environment-step budget instead of the same number of episodes?
  • With fixed total archive capacity and no detection-triggered resets, does physics indexing alone improve on a global archive under the same Bayesian selection?
  • How do the baselines compare if they are also allowed test-time selection, for example among multiple saved checkpoints with a bandit?
  • On HalfCheetah, what is ENA's absolute forgetting compared with SAC, given that SAC learns much higher returns?
  • Does the chirp detector hold up on physics changes outside the environments used to select its excitation and features?
  • How does ENA compare with MBCD or TeLAPA on the same streams?

Code2Video uses agent-authored 3D proxies to guide video generation, enabling explicit control over structure and dynamics. However, how to construct proxies and prompts for optimal downstream quality remains underexplored, as do the capabilities and limitations of this paradigm. Through controlled experiments and blinded human evaluation, we study proxy granularity, physics simulation, and prompting. The results reveal task-dependent trade-offs: coarse proxies favor natural motion, while medium-grained proxies favor consistency. Physics simulation and proxy-aligned prompting further improve average quality. Guided by these findings, we introduce Coda-DaVinci, a task-adaptive agentic framework for Code2Video generation that selects between proxy-guided and direct generation, adapts modeling granularity to each entity, and coordinates simulation with proxy-grounded prompting. Its multi-agent harness preserves scene assets and guides repairs through visual inspection and independent review. Experiments demonstrate leading instruction following on VTR and scene consistency on R2M, improved physical plausibility over direct generation, and higher visual realism than the compared Code2Video baselines. Long-video evaluations further show gains in event structure and consistency across successive interactions, while revealing limitations in transition stability and adherence to scene settings.

Yiyan Ji, Qixun Wang +11

Agents based on large language models (LLMs) can now delete files and send payments on behalf of their users. News media reports routinely describe failures of these agents (e.g., Replit's coding agent deleting a production database), but a report often only says what went wrong and rarely (in)validates the incident or finds its root cause. Incident catalogs collect these reports, and benchmarks record whether agents fail, but neither replicates an incident nor tests the cause its report might identify. In this paper, we present ROGUE-AI (Replication of Agentic Security Events: Audit and Investigation), a multi-agent framework that closes the above gap and rebuilds each reported incident offline together with a matched control that lacks only the blamed cause, and runs agents on both. Our results suggest that ROGUE-AI is able to reproduce the reported harm in 89% of its 108 attempted replications. Agents built on five current GPT, Claude, Gemini and DeepSeek models made the reported mistake in 18% to 33% of runs, against at most 1% without the blamed cause. Of the 72 incidents with real-world harm, 62 had no underlying attacker, and in these, agents did what their task asked for but missed an ordinary hazard, such as a database shared with the live site. Replications without an attacker caused more harm than replications of prompt injection for all five models. Defenses against prompt injection look for injected instructions, but these replications contain none. Further, a warning after each tool output that names the hazard and its consequence lowered harm for all five models, whereas a note of the same length that does not mention the hazard left most of the harm in place. Built from real incidents, the ROGUE-AI replications form a reusable audit framework that vendors can rerun before each new release.

Mengxiao Wang, Mahbub Alam +2

Pretrained language models (LMs) remain unreliable on exact multi-digit arithmetic without gradient fine-tuning or external tool interpreters. We introduce closed-form, zero-gradient circuit-grafting primitives that compile deterministic computation directly into pretrained LM weights using zero training examples and zero runtime hooks, while bounding per-layer normalization drift and preserving natural-language (NL) competence. Our construction combines three analytical mechanisms: (1) two-layer Mellin-GLU quadrature over bias-free SwiGLU/GeGLU layers, turning unknown per-token RMSNorm scaling into translation along a log-spaced gate grid ($>250\times$ dynamic range) with IEEE 754 error bounds; (2) cross-token scale cancellation via signed Chebyshev Indefinite-RoPE self-sink attention and memoryless SiLU-antisymmetric GatedDeltaNet shifts; and (3) axis-aligned width extension ($d \to d+K$) with $\sqrt{d/(d+K)}$ norm compensation, Base-1000 canonical BPE emission, and differential $(+254,-256)$ tied unembeddings. An automated 5-stage compiler synthesizes 4-8 digit signed $\{+,-,\times\}$ circuits in constant $\mathsf{TC}^0$ depth alongside 3-literal Boolean (100%, $N=1{,}728$; plus 42-layer Gemma 4 E4B audits) and 5-bit Majority/Parity (100%) circuits. On 6-digit arithmetic ($N=1{,}200$, BF16), IBM Granite 4.2 3B (1.07M-1.33M active non-zeros, $\le 0.036\%$) and Alibaba Qwen 3.5 4B (2.25M-2.39M non-zeros, $\le 0.057\%$) reach 99.17% and 100.00% ALU-prefix accuracy (100% with an FP32 sidecar) and 98%-100% mid-sentence resumption, outperforming 12.8K-sample LoRA (42.2% / 61.7%, 0% inline continuation). A factorized block-sparse sidecar holds A100 memory at 7.89 / 10.65 GiB (4.60 / 8.14 GiB under INT8/INT4 + FP32 sidecar) while passing all 14/14 pre-registered $>2$K-token Tango splits ($N=4{,}392$; 0/11,134 OOD triggers).

Tanmay Sah

The Harness Hub newsletter

New harness companies, projects, releases and the monthly State of Harness report, in your inbox.