Harness Hub
All papers

Efficient Modeling of Multi-Agent Systems via Structure-Aware Physics-Informed Learning

Yulun Wu, Nicola Bastianello, Karl Henrik Johansson, Matthieu Barreau

Verdict: Incremental

The paper is a reasonable engineering extension of the authors' FALM-PINN to networked ODEs, and the matrix-free solver is a sensible use of graph sparsity. Several gaps limit the evaluation: - the inference baselines are not compute-matched; - there is no comparison against a standard ODE integrator; - the efficiency claim is benchmarked only against the authors' own dense solver; - one baseline appears degenerate; - the identification problems have only 8 nodes. The paper has no bearing on AI agent harnesses.

  • Novelty

    2 out of 5
  • Rigor

    2 out of 5
  • Reproducibility

    4 out of 5
  • Relevance to harnesses

    1 out of 5

TL;DR

A physics-informed neural network for coupled ODE networks: Lorenz rings, Lotka-Volterra site networks and Rössler oscillators on hypergraphs. 'Multi-agent' here means dynamical-system nodes, not AI agents. The method extends the authors' FALM-PINN with a shared temporal basis and a matrix-free Levenberg-Marquardt/CG coefficient solve. It is irrelevant to agent harness builders.

Adversarial review by Harness Hub · Nov 18, 2026 · read full text (truncated) · Suggest a correction

Original abstract (authors’ text, select to expand)

Modeling large multi-agent systems poses challenges for trajectory inference and parameter identification, as network size and coupling increase computational demands and partial observations require reconstructing unobserved states. We propose MAS-PINN, a neural-network architecture with a tailored training algorithm for a physics-informed learning framework that leverages the network topology and the agents' similarity to reduce computational and memory costs of simulating and identifying multi-agent systems. Its architecture represents agent trajectories through shared temporal features and agent-specific coefficients. Training alternates between learning features, fitting coupled trajectories, and estimating physical parameters. A matrix-free update exploits local interactions to fit trajectories without forming large global matrices. Experiments on Lorenz rings, Lotka--Volterra networks, and Rössler oscillators show improved trajectory and parameter accuracy over baselines, including inference in large systems and identification under partial observations or unknown topology, alongside substantial computational savings in coefficient fitting.

Filed undermulti-agentmemoryagents

Summary

MAS-PINN represents every node trajectory as agent-specific linear coefficients over one shared Fourier-feature basis. Each outer iteration (Table 4) does three things: - one Adam step on the basis parameters; - one Levenberg-Marquardt step on the coefficients; - for identification only, a least-squares solve for the physical parameters. The LM normal equations are solved approximately by conjugate gradients, with at most 100 iterations. The Jacobian-vector products follow the graph structure, so the dense Jacobian and normal matrix are never formed. Proposition 3 bounds the cached linear solve at O(N) work per operator application and O(N) storage, for sparse graphs with fixed P, d, N_f and N_d. The authors state explicitly that this bound does not cover full training. Experiments: - Inference on a Lorenz ring with N in {50,100} over [0,1]. - Identification of a Lotka-Volterra network with N=8 and four of eight sites observed. Results are shown only as box plots in Figure 2. - Identification of Rössler hypergraph topology and weights with N=8. - Timing of the coefficient solver against the authors' own dense-Cholesky LM, scaling the solver alone up to N=256 (Table 8).

Key findings

  • Table 1, Lorenz N=100: trajectory relative L2 error is (2.2 ± 0.9)×10^-4 for MAS-PINN, (1.6 ± 0.7)×10^-1 for PirateNets and (5.6 ± 0.9)×10^-1 for vanilla PINN. MAS-PINN uses 1×10^4 iterations versus 2×10^5 for both baselines (Table 5).
  • Table 3, Rössler hypergraph at 0.5% noise: MAS-PINN w error is (4.5 ± 2.3)×10^-3 versus MHE (3.2 ± 1.6)×10^-2 and HyperPINN (4.7 ± 2.6)×10^-1. MAS-PINN and MHE both reach AUC 1.00; HyperPINN reaches 0.96 ± 0.02.
  • Table 2, Lotka-Volterra reference setting (N=8, N_f=1600, P=129): matrix-free LM with 100 CG iterations takes 3.64 s and 32 MB peak memory, versus 77.77 s and 3194 MB for dense LM with Cholesky.
  • Table 8: matrix-free LM time grows from 3.64 s at N=8 to 41.35 s at N=256, and LM-stage memory from 5.7 MB to 216.1 MB. Dense LM was not run for N ≥ 32.
  • Lotka-Volterra identification results (Figure 2) appear only as box plots. The text claims lower errors than the baselines for A, w and trajectories at all three noise levels, but gives no numbers.

The paper at a glance

01Shared MLP + randomFourier features ->basis psi(t)02Agent-specificcoefficients W_i ->x_i(t) = W_i^Tpsi(t)03Adam step on basisparameters04Matrix-free LM stepon coefficients viatruncated CG withgraph-structured Jv/ J^T u05Least-squaresupdate of physicalparameters(identificationonly)
  1. Shared MLP + random Fourier features -> basis psi(t)
  2. Agent-specific coefficients W_i -> x_i(t) = W_i^T psi(t)
  3. Adam step on basis parameters
  4. Matrix-free LM step on coefficients via truncated CG with graph-structured Jv / J^T u
  5. Least-squares update of physical parameters (identification only)
Fig. 1 — How the paper works, step by step (arXiv:2611.21404).
Matrix-free, 30 CG iterations (this paper)1.2 sMatrix-free, 60 CG iterations (this paper)2.25 sMatrix-free, 100 CG iterations (this paper)3.64 sDense LM, Cholesky77.77 s
Fig. 2 — Coefficient-update (LM) time over 100 outer iterations, Lotka-Volterra reference setting, lower is better. Source: Table 2, arXiv:2611.21404.
Data table
Coefficient-update (LM) time over 100 outer iterations, Lotka-Volterra reference setting
Methods
Matrix-free, 30 CG iterations (this paper)1.2 s
Matrix-free, 60 CG iterations (this paper)2.25 s
Matrix-free, 100 CG iterations (this paper)3.64 s
Dense LM, Cholesky77.77 s

Why it matters

This paper is irrelevant to people building AI agent harnesses. Its 'multi-agent systems' are nodes in coupled ordinary differential equations: chaotic Lorenz sites, ecological species patches and nonlinear oscillators. It involves no language-model agents, tool loops, memory, orchestration or evals. The transferable ideas are generic numerical-optimization techniques: a shared basis with per-node coefficients, and inexact Gauss-Newton with Krylov solves whose Jacobian products follow the graph like message passing. These matter to people fitting physics-informed surrogates of networked dynamical systems in control, power systems or ecology. Nothing here changes how one would design, run or evaluate a coding agent or an agent framework.

Issues (10)

  • High severity

    The inference comparison is not compute-matched. The claim of 'fewer training epochs' hides unequal per-step cost. MAS-PINN runs 1×10^4 outer iterations, each with an Adam step plus a full LM step of up to 100 CG iterations. The baselines run 2×10^5 Adam steps. Architectures also differ: MAS-PINN has hidden depth 3, the baselines depth 5. No wall-clock or FLOP comparison against vanilla PINN or PirateNets is reported. The accuracy gap therefore cannot be separated from the second-order solver.

  • High severity

    The abstract claims reduced cost for 'simulating' multi-agent systems, but inference is never compared against numerical integration. The reference trajectory is itself produced by DOP853 at tolerance 10^-12. For known parameters and initial state, a standard integrator is the natural baseline in both time and accuracy, and the paper reports no runtime for it.

  • High severity

    'Substantial computational savings' are measured only against the authors' own dense LM, which explicitly forms the Jacobian and normal matrix and uses Cholesky factorization. Realistic alternatives would be sparse direct factorization or an off-the-shelf Jacobian-free Newton-Krylov method. No runtime is reported against the actual baselines (PirateNets, HyperPINN, MHE). Total end-to-end training time is never given; Table 2 covers only 100 outer iterations.

  • Medium severity

    The HyperPINN baseline appears to fail rather than merely underperform. Its errors are identical at all three noise levels: w (4.7 ± 2.6)×10^-1, v (9.9 ± 0.1)×10^-1, AUC 0.96 ± 0.02. A v error near 1 is consistent with predicting near-zero three-body weights. No tuning effort for the baselines is described, so the margin over HyperPINN says little. MHE, the stronger baseline, also reaches AUC 1.00.

  • Medium severity

    The Lotka-Volterra identification claim cannot be checked from the text. Results are shown only as box plots in Figure 2, with no numeric table, unlike the other two experiments.

  • Medium severity

    'Inference in large systems' is overstated. Inference uses at most N=100 scalar Lorenz sites, with d=1, over a short horizon of [0,1]. Both identification problems use N=8. Only the cached coefficient-solve timing is scaled to N=256, and accuracy at that scale is never reported.

  • Medium severity

    There is no ablation against the authors' prior FALM-PINN. It is therefore unclear how much accuracy comes from the shared-basis design versus the inherited Fourier-feature plus LM scheme.

  • Low severity

    The cost bound assumes fixed P, N_f and N_d. The paper says the bound does not assume k_CG is independent of N, yet the scaling runs cap CG at 100 iterations. Solve quality at larger N under that cap is therefore unverified. Identifiability and observability are assumed (Remark 1) and not checked for the partially observed case.

  • Low severity

    Memory figures at the same N=8 setting differ between tables. Matrix-free peak is 32 MB in Table 2 versus 5.7 MB in Table 8; dense is 3194 MB versus 3169 MB. This is explained by different measurement scopes (full iteration versus LM stage), but makes the headline ratio depend on which table is cited. No timing variance is reported.

  • Low severity

    MHE runs on an Intel i9 CPU while the neural methods run on an RTX 4090. This rules out cross-method efficiency comparison, and none is offered.

Questions for the authors

  • What wall-clock time does MAS-PINN need to reach the Table 1 errors, compared with PirateNets and vanilla PINN on the same GPU?
  • How does MAS-PINN compare in time and accuracy with integrating the Lorenz system directly with DOP853, which already produces the reference solution?
  • Why are HyperPINN's errors invariant to noise level, and were its sparsity schedule and penalties retuned for this setup?
  • What are the numeric Lotka-Volterra errors for A, w and trajectories behind Figure 2?
  • Do trajectory and parameter accuracy hold at N=256 under the 100-iteration CG cap?
  • How do results change against FALM-PINN without the shared basis, or against a sparse direct solver instead of dense Cholesky?

Code2Video uses agent-authored 3D proxies to guide video generation, enabling explicit control over structure and dynamics. However, how to construct proxies and prompts for optimal downstream quality remains underexplored, as do the capabilities and limitations of this paradigm. Through controlled experiments and blinded human evaluation, we study proxy granularity, physics simulation, and prompting. The results reveal task-dependent trade-offs: coarse proxies favor natural motion, while medium-grained proxies favor consistency. Physics simulation and proxy-aligned prompting further improve average quality. Guided by these findings, we introduce Coda-DaVinci, a task-adaptive agentic framework for Code2Video generation that selects between proxy-guided and direct generation, adapts modeling granularity to each entity, and coordinates simulation with proxy-grounded prompting. Its multi-agent harness preserves scene assets and guides repairs through visual inspection and independent review. Experiments demonstrate leading instruction following on VTR and scene consistency on R2M, improved physical plausibility over direct generation, and higher visual realism than the compared Code2Video baselines. Long-video evaluations further show gains in event structure and consistency across successive interactions, while revealing limitations in transition stability and adherence to scene settings.

Yiyan Ji, Qixun Wang +11

Agents based on large language models (LLMs) can now delete files and send payments on behalf of their users. News media reports routinely describe failures of these agents (e.g., Replit's coding agent deleting a production database), but a report often only says what went wrong and rarely (in)validates the incident or finds its root cause. Incident catalogs collect these reports, and benchmarks record whether agents fail, but neither replicates an incident nor tests the cause its report might identify. In this paper, we present ROGUE-AI (Replication of Agentic Security Events: Audit and Investigation), a multi-agent framework that closes the above gap and rebuilds each reported incident offline together with a matched control that lacks only the blamed cause, and runs agents on both. Our results suggest that ROGUE-AI is able to reproduce the reported harm in 89% of its 108 attempted replications. Agents built on five current GPT, Claude, Gemini and DeepSeek models made the reported mistake in 18% to 33% of runs, against at most 1% without the blamed cause. Of the 72 incidents with real-world harm, 62 had no underlying attacker, and in these, agents did what their task asked for but missed an ordinary hazard, such as a database shared with the live site. Replications without an attacker caused more harm than replications of prompt injection for all five models. Defenses against prompt injection look for injected instructions, but these replications contain none. Further, a warning after each tool output that names the hazard and its consequence lowered harm for all five models, whereas a note of the same length that does not mention the hazard left most of the harm in place. Built from real incidents, the ROGUE-AI replications form a reusable audit framework that vendors can rerun before each new release.

Mengxiao Wang, Mahbub Alam +2

Pretrained language models (LMs) remain unreliable on exact multi-digit arithmetic without gradient fine-tuning or external tool interpreters. We introduce closed-form, zero-gradient circuit-grafting primitives that compile deterministic computation directly into pretrained LM weights using zero training examples and zero runtime hooks, while bounding per-layer normalization drift and preserving natural-language (NL) competence. Our construction combines three analytical mechanisms: (1) two-layer Mellin-GLU quadrature over bias-free SwiGLU/GeGLU layers, turning unknown per-token RMSNorm scaling into translation along a log-spaced gate grid ($>250\times$ dynamic range) with IEEE 754 error bounds; (2) cross-token scale cancellation via signed Chebyshev Indefinite-RoPE self-sink attention and memoryless SiLU-antisymmetric GatedDeltaNet shifts; and (3) axis-aligned width extension ($d \to d+K$) with $\sqrt{d/(d+K)}$ norm compensation, Base-1000 canonical BPE emission, and differential $(+254,-256)$ tied unembeddings. An automated 5-stage compiler synthesizes 4-8 digit signed $\{+,-,\times\}$ circuits in constant $\mathsf{TC}^0$ depth alongside 3-literal Boolean (100%, $N=1{,}728$; plus 42-layer Gemma 4 E4B audits) and 5-bit Majority/Parity (100%) circuits. On 6-digit arithmetic ($N=1{,}200$, BF16), IBM Granite 4.2 3B (1.07M-1.33M active non-zeros, $\le 0.036\%$) and Alibaba Qwen 3.5 4B (2.25M-2.39M non-zeros, $\le 0.057\%$) reach 99.17% and 100.00% ALU-prefix accuracy (100% with an FP32 sidecar) and 98%-100% mid-sentence resumption, outperforming 12.8K-sample LoRA (42.2% / 61.7%, 0% inline continuation). A factorized block-sparse sidecar holds A100 memory at 7.89 / 10.65 GiB (4.60 / 8.14 GiB under INT8/INT4 + FP32 sidecar) while passing all 14/14 pre-registered $>2$K-token Tango splits ($N=4{,}392$; 0/11,134 OOD triggers).

Tanmay Sah

The Harness Hub newsletter

New harness companies, projects, releases and the monthly State of Harness report, in your inbox.