Best practices for adopting open-source agent harnesses

Stars tell you about attention, not about fitness. Check these before anyone installs anything:

License. Both of the big personal-agent harnesses are MIT (openclaw/openclaw and NousResearch/hermes-agent, per the GitHub API). That is permissive, but check plugin and skill marketplaces separately. Third-party skills carry their own licenses and their own risk.

Governance and bus factor. Find out who merges, and who handles security reports. OpenClaw's SECURITY.md says security work is "shared across a number of OpenClaw maintainers, including engineers and security researchers from organizations such as NVIDIA and Tencent." Coverage at DeepLearning.AI notes that OpenClaw's inventor later joined OpenAI. If a single founder leaves a project, ask what happens next. OpenSSF Scorecard's Contributors check (contributors from at least two organizations) and Maintained check put numbers on this.

Release cadence. Fast is not automatically good. As of 2026-10-27, OpenClaw was shipping several calendar-versioned releases a day across parallel lines (v2026.10.8, v2026.9.35, v2026.10.10-beta.4). Hermes shipped five patch releases (v0.22.1 to v0.22.5) between 14 and 23 October. Either pace means you need a pinning strategy (§8), not a habit of tracking main.

Security posture. Run OpenSSF Scorecard (scorecard --repo=github.com/<owner>/<repo>, or the viewer at scorecard.dev/viewer/?uri=...). Look closely at Branch-Protection, Code-Review, Dangerous-Workflow, Pinned-Dependencies, Signed-Releases, Token-Permissions and Vulnerabilities. Then read the project's trust model, which matters more than its CVE count:

• OpenClaw describes itself as "local-first agent infrastructure for trusted operators; it is not designed as a shared multi-tenant boundary between adversarial users on one gateway." It adds that "Session ownership, visibility, and presence are usability features, not security boundaries," and that "prompt-injection-only chains" are usually not treated as vulnerabilities.
• Hermes is just as blunt: "The only security boundary against an adversarial LLM is the operating system. Nothing inside the agent process constitutes containment — not the approval gate, not output redaction, not any pattern scanner, not any tool allowlist." It also notes that the default terminal backend "runs commands directly on the host."

These statements are honest and useful. They tell you that you are responsible for isolation. If your plan was to share one agent gateway across a team, both projects are telling you not to.

Advisory volume, read carefully. The GitHub API lists at least 1,000 published security advisories on openclaw/openclaw (the first page of 100 has 51 high, 44 medium and 5 low). NousResearch/hermes-agent lists zero published repository advisories. A high count can mean a lot of bugs, a large active security programme, or both. A zero can mean few bugs, or that disclosures happen somewhere else. Ask the maintainers before you draw a conclusion either way.

Model/provider lock-in. Check whether swapping providers works in practice and not only in config. One r/LocalLLM commenter said the reason they preferred Hermes was that switching the backend model "actually takes effect," unlike their OpenClaw setup. Test this yourself with your own providers.

MCP and skills support. Most harnesses now speak MCP and the SKILL.md format (Hermes uses the standard SKILL.md format per DeepLearning.AI). That helps portability. It also makes it easier to mix tools from different trust levels, which leads straight to §7.

Anthropic's eval guidance is the most practical starting point we've found. "20-50 simple tasks drawn from real failures is a great start." Turn bug reports into test cases. "It's often better to grade what the agent produced, not the path it took." Pick the right metric. pass@k (at least one success in k tries) suits developer tools. pass^k (all k succeed) suits anything customer-facing. Read the transcripts, because that's how you tell an agent mistake from a broken grader.

For harness selection specifically:

• Run the same task set on two or three candidate harnesses with the same model. You're measuring the harness.
• Track tokens and wall-clock time alongside pass rate. One Hermes user said it "eats ridiculous amounts of tokens, especially cached." Another went back to pi because it was slow and token-hungry. You want to find that out in an eval, not on an invoice.
• Include small/local models if you plan to use them. One commenter found small models on OpenClaw, Agent Zero or Hermes "lose focus too easily" for web-app development.

OWASP LLM06 (Excessive Agency) names three root causes: excessive functionality, excessive permissions and excessive autonomy. Its mitigations fit harness adoption directly:

• Narrow tools over broad ones. Avoid open-ended shell and URL-fetch tools where a purpose-built tool will do.
• Authorization lives downstream. "Implement authorization in downstream systems rather than relying on an LLM to decide if an action is allowed or not." Use the user's own identity with minimal OAuth scopes.

Then put an OS boundary around the agent. Claude Code's sandbox docs are a good model of what "real" looks like, and of where it stops:

• Filesystem and network isolation, because they are independent layers. Writes are limited to the working directory. Network traffic goes through a proxy whose allowlist "starts empty."
• The defaults still have gaps you need to close. Reads cover "the entire computer, except certain denied directories" and "this default still allows reading credential files." Add denyRead for ~/ and deny ~/.ssh and token env vars.
• Know what sits outside the boundary: "Claude's file tools, MCP servers, and hooks run outside it". Excluded commands run "with your full access."
• Fail closed. By default, if the sandbox can't start, commands run unsandboxed. Set failIfUnavailable: true and allowUnsandboxedCommands: false.

For Hermes, choose a non-default terminal backend (container, remote host or cloud sandbox). Its own policy explains that this confines shell and file tools but not code running in the agent's Python process, including skills and plugins. Practitioners say the same in plainer words. On r/MachineLearning, one user ran OpenClaw in a VM but would not run it on their main PC: "Too much risk."

Network egress is where data actually leaves. The MCP security best practices recommend egress proxies (e.g. Stripe's Smokescreen), blocking private ranges including 169.254.0.0/16 cloud metadata, and not hand-rolling IP validation.

Never expose the control plane. In early 2026, secondary coverage of SecurityScorecard and Censys research reported tens of thousands of internet-exposed OpenClaw instances, and CVE-2026-25253 (CVSS 8.8, one-click RCE via WebSocket hijacking) was patched in 2026.1.29. We could not open the primary reports, so treat the exact counts as unverified. The lesson holds regardless: bind to localhost or a private network, and put auth in front.

OWASP LLM01 and LLM06 both say to require human approval for high-impact actions. In practice:

• Sort actions into tiers. Read-only actions run freely. Reversible writes inside the sandbox run freely. Anything external, irreversible or touching money or production goes through a prompt or a PR.
• Use content-scoped rules. Claude Code's Bash(git push *)-style ask rules "still force a prompt even for sandboxed commands." Most harnesses have an equivalent.
• Make the approval screen show the whole action. Invariant Labs found that the confirmation dialog often hides the full tool arguments. The MCP spec requires clients to "show the exact command that will be executed, without truncation" before adding a local server.
• For coding agents, the PR is the gate. Agent output goes through the same CI, review and branch protection as human output.

Remember Hermes's warning: an approval gate is a heuristic, not a boundary. It reduces risk, and it works best when the sandbox is still there behind it.

The cost stories are the most consistent complaint across communities:

• Notebookcheck documented OpenClaw's default 30-minute heartbeat resending ~120,000 tokens of context per check on Opus, about $0.75 per request. One user's overnight idle run cost about $18.75. The article puts a week of idle checks at about $250.
• One r/LocalLLM poster tracked 30 days of OpenClaw spend and cut it from $340 to $112 with prompt caching, batching, output caps and moving simple tasks off GPT-4.1: "I had no idea where my money was going before I actually tracked it." Commenters added: use small worker models for mechanical tasks, "use as much scripting as you possibly can," kill looping calls, audit logs.

What to put in place before go-live:

1. Hard spend limits at the provider or gateway, per agent and per day.
2. Per-call tracing: model, tokens in/out/cached, tool calls, latency and session ID. The OpenTelemetry GenAI semantic conventions (now maintained in open-telemetry/semantic-conventions-genai) define agent spans, metrics, events and MCP conventions. Use them so you aren't locked into one vendor's tracing.
3. Alerts on loops: repeated identical tool calls and long sessions that never reset.
4. Look at scheduled or background behaviour (heartbeats, cron, title generation) separately. That's where idle money goes.

OWASP notes that monitoring and rate-limiting "don't prevent the problem but can limit damage and help detect misuse early." The same is true for cost.

Anthropic's context-engineering guidance treats context as "a finite resource with diminishing marginal returns" (context rot). The patterns that work:

• Compaction. Summarize the conversation as it nears the limit. Clearing old tool results is the "lightest touch" option.
• Structured notes. Keep a NOTES.md or to-do file outside the window.
• Sub-agents. Run focused work in clean windows that return "a condensed, distilled summary."
• Just-in-time retrieval. Keep file paths, queries and links rather than preloading everything.

Hermes builds much of this in. Per DeepLearning.AI, it keeps two memory files (user preferences, and workflows and lessons learned), dedupes and merges entries, has a searchable conversation DB, auto-creates skills after long tasks, and runs a Curator that archives skills unused for 90 days. Users praise it: "It does memory particularly well and improves itself." One practitioner's advice is worth repeating: keep memory scoped and versioned, test skills like code, and sandbox tool access. Self-written memory and skills persist across sessions, so an injected instruction saved to memory comes back every time. Review memory diffs the way you'd review config.

Assume prompt injection will happen. OWASP says it is "unclear" whether foolproof prevention exists. The architecture-level defences:

• Break the lethal trifecta. Simon Willison names the three ingredients: private data, untrusted content, and an exfiltration channel. "The only way to stay safe there is to avoid that lethal trifecta combination entirely." MCP makes it easy to assemble all three by accident. Guardrail products that catch "95% of attacks" are not a security boundary.
• Constrain after untrusted input. Once an agent has read untrusted content, limit what it can do next.
• Segregate and label external content, validate outputs with deterministic code, and red-team the agent "as an untrusted user" (OWASP LLM01).
• Tool poisoning and rug pulls. Invariant Labs showed hidden instructions in MCP tool descriptions, and servers that change descriptions after approval. Pin server versions, hash tool descriptions and alert on change, and enforce data-flow boundaries between servers.
• Treat skill marketplaces like npm on a bad day. A Koi Security audit (via secondary coverage) flagged 341 of 2,857 ClawHub skills as malicious, 335 of them from one campaign pushing Atomic Stealer through fake prerequisites. At the time, a publisher needed only a GitHub account at least a week old. OpenClaw responded with VirusTotal scanning and daily rescans. As r/MachineLearning commenters put it, skills are "context the model trusts," and "exploitation moves from code execution to intent manipulation." Combinations of individually benign skills can be dangerous. Keep an internal allowlist of vetted, pinned skills and deny everything else by default at the orchestrator.
• MCP auth hygiene. No token passthrough ("MUST NOT accept any tokens that were not explicitly issued for the MCP server"). Use progressive, least-privilege scopes rather than *.

• Pin exact harness versions, MCP server versions and skill commits. Commit a lockfile.
• Upgrade on a schedule, not on every release. Re-run your eval suite (§2) and a small injection suite before promoting a version.
• Subscribe to the repo's security advisories and patch out-of-band for high-severity ones. The CVE-2026-25253 fix landed in under 24 hours, so being able to roll it out fast matters.
• Watch for tool-description and permission diffs on upgrade, not just code diffs.
• Apply Scorecard's Pinned-Dependencies and Token-Permissions standards to your own harness deployment repo.

Both major projects say they want focused, reproducible reports and PRs, and neither runs a bug bounty. OpenClaw wants "a current, reproducible boundary bypass with demonstrated impact," with version and commit SHA. Hermes wants file path and line range plus "which trust boundary in §2 is crossed." Read the trust model before filing. Heuristic bypasses that are out of scope for the security channel are still welcome as regular issues or PRs (Hermes §3.2). Upstream your sandbox configs, eval tasks and cost-guard fixes as well. Those help everyone, and they reduce the number of patches you maintain on a fork.

Anthropic's long-standing advice is to find "the simplest solution possible," because frameworks "often create extra layers of abstraction that can obscure the underlying prompts," and to "start by using LLM APIs directly." Agents earn their place on open-ended problems where the number of steps is unpredictable.

Adopt when you need breadth fast (messaging gateways, many tools, memory, schedulers), your use is single-tenant and matches the project's trust model, and you can wrap it in your own sandbox, egress and cost controls.

Build (or use a thin SDK) when you need multi-tenant isolation the projects explicitly don't provide, tight audit and authorization integration, or predictable cost. One r/LocalLLM commenter argued that a fully custom agent "built with the right know-how will beat it". Others point out the opposite risk: you end up maintaining a harness instead of shipping product.

Community scorecard: OpenClaw vs Hermes (as reported, not verified by us)

Whichever you pick: OS-level sandbox, empty egress allowlist, pinned skills, spend caps, traces, and your own evals.