CLI 才是裁判：把組織政策編譯成 agents-cli 擴展

Google shipped an Extension System for agents-cli in v1.5.0. One YAML file in a directory, picked up by the CLI, and suddenly agents-cli deploy runs your compliance check instead of the built in. No fork, no wrapper scripts your team forgets to call, no divergent copy to rebase every release.

The announcement frames it as the end of framework lock in. Six of the twenty four odd commands in the CLI assume the ADK runtime. The rest, deploy, infra, publish, login, are framework agnostic. The extension system lets a LangChain project replace the six and inherit the other eighteen, which is a genuinely good design.

But it leaves one thing to you: writing the extension.

That is not a small thing. The manifest is strict. run: is an argv vector executed with no shell, so any chaining has to live in a script. Some commands are reserved and cannot be claimed. You can override eval.generate but not the eval group. Misspell a key and the loader tells you, with a did you mean, but only after you have read the docs.

I built a tool that closes that gap, and the interesting part is not the model call. It is the division of labour.

The model writes. The CLI judges.

Repo: the-override. Node server, React UI, agents-cli 1.11.0 installed from PyPI, any OpenAI compatible model endpoint. That is the whole stack.

The shape of the thing

you type a policy
      |
      v
model returns JSON: commands, scripts, demo command
      |
      v
server renders agents-cli-extension.yaml   (the model never writes YAML)
      |
      v
agents-cli extension add local@... --yes   (the real binary, in a throwaway project)
      |
      v
extension list | info | deploy --help      (real stdout, streamed to the browser)
      |
      v
agents-cli deploy                          (the gate runs, then exits non zero)

Every line the UI shows under "verdict" is stdout or stderr from the binary on the machine. There is no fixture, no recorded cassette, no friendly paraphrasing. If the CLI is not installed, the app tells you the pip command instead of pretending.

Why not let the model write the YAML?

Because then you are debugging YAML generated by a language model, which is one of the least pleasant debugging experiences available.

The manifest has five or six top level keys, and the schema uses extra="forbid", so a typo is a hard failure with a helpful message, but only after you have burned a model call. Meanwhile the actual decision the model needs to make is small: which command does this policy belong to, what does the wrapper do, which state file does it read.

So the split is:

• model: name, description, rationale, commands.override, commands.add, script bodies, demo command
• server: everything about spelling

function renderManifest(spec) {
  const name = String(spec.name || '').trim()
  if (!/^[A-Za-z0-9._-]+$/.test(name)) throw new Error(`bad extension name ${JSON.stringify(name)}`)
  const lines = [
    'schema: agents-cli-extension/v1alpha1',
    `name: ${q(name)}`,
    `description: ${q(spec.description || '')}`,
    'requires:',
    '  agents_cli: ">=1.5,<2"',
    '  on_incompatible: warn',
    'commands:',
  ]
  for (const kind of ['override', 'add']) {
    const entries = Object.entries(spec.commands?.[kind] || {})
    if (!entries.length) continue
    lines.push(`  ${kind}:`)
    for (const [cmd, entry] of entries) {
      lines.push(`    ${q(cmd)}:`)
      lines.push(`      run: [${(entry.run || []).map(q).join(', ')}]`)
      if (entry.description) lines.push(`      description: ${q(entry.description)}`)
    }
  }
  return lines.join('\n') + '\n'
}

q() is JSON.stringify. A double quoted YAML scalar accepts exactly what JSON produces, so the server needs no YAML dependency at all. Forty lines replaces a parser, a serialiser, and a class of failures.

This is the pattern I keep coming back to: make the model produce a small typed decision, then let boring code turn it into the artifact. Not because models cannot write YAML, but because a wrong YAML key fails in a place that is expensive to debug and cheap to prevent.

Give the model the real command surface

A model writing an extension needs to know which commands exist. You can paste the docs into the prompt and watch them drift within one release. Or you can ask the binary.

const version = (await cli(['--version'], ROOT)).out.trim().replace(/^.*?version\s+/i, '')
const help = (await cli(['--help'], ROOT)).out
const top = []
for (const line of help.split('\n')) {
  const m = line.match(/^ {2}([a-z][a-z0-9-]+) {2,}/)
  if (m && !['Options', 'Commands', 'Source'].includes(m[1])) top.push(m[1])
}

On 1.11.0 that yields 17 top level commands, plus subcommands harvested the same way from eval, scaffold, publish, create and infra. Those dotted names go into the prompt, so the model picks from eval.generate and scaffold.enhance rather than inventing eval.run-evals. The same data feeds the header chip in the UI, which is why the note says 17 commands and not "a bunch".

The reserved list is hardcoded, and that is deliberate. It mirrors RECOVERY_COMMANDS in the CLI's loader: install, extension, info, login, setup, update. These are the commands you use to inspect and repair the CLI, so handing them to an extension would be a trap. The model is told not to claim them, and if it does anyway, the CLI refuses and the UI shows that refusal. Which brings me to the best part of the demo.

The refusal is the feature

Ask the tool to "override the login command so it forces corporate SSO" and you get this, for real:

$ agents-cli extension add local@./sso-gate --yes
❌ Error: extension 'sso-gate' claims 'login', which cannot be overridden
   (it is reserved for inspecting, setting up or repairing agents-cli)

A demo that only shows success is a brochure. A demo that shows the tool being told no, by the authority it defers to, is a product. Every successful compile now ends with a refusal case, because reviewers trust the boundary once they have seen it enforced.

The same applies to a subtler mistake that models make constantly: overriding a group instead of a subcommand. commands.override.eval is refused, commands.override["eval.generate"] is fine, and the difference is exactly the kind of thing a human skims past.

Streaming a 40 second wait

Four agents-cli invocations is roughly 30 to 45 seconds, because each one is a cold Python start with cloud SDK imports. That is not a performance problem worth solving with a daemon, but it is a UX problem if the panel is blank.

So /api/verify answers in NDJSON: one JSON object per line, flushed as each command finishes.

res.writeHead(200, { 'content-type': 'application/x-ndjson', 'cache-control': 'no-cache' })
const send = (o) => res.write(JSON.stringify(o) + '\n')
const result = await verify(spec, (step) => send({ type: 'step', step }))
send({ type: 'done', ...result })
res.end()

On the client, a reader loop. Not EventSource, because the request is a POST with a body and EventSource cannot do that:

const reader = r.body.getReader()
const dec = new TextDecoder()
let buf = ''
for (;;) {
  const { value, done } = await reader.read()
  if (done) break
  buf += dec.decode(value, { stream: true })
  const lines = buf.split('\n')
  buf = lines.pop() || ''            // keep the partial line for the next chunk
  for (const line of lines) {
    if (!line.trim()) continue
    const ev = JSON.parse(line)
    if (ev.type === 'step') setVSteps((prev) => [...prev, ev.step])
    else if (ev.type === 'done') done = ev
  }
}

The buf = lines.pop() detail is the one people get wrong. Chunks do not respect line boundaries, so the last split element is almost always a partial JSON object. Holding it back is three characters and saves a whole class of intermittent parse errors.

Measured here: the RUNNING banner appears at 9.3 seconds with the install step already done, then ACCEPTED at 29.9 seconds with five steps. Same wall clock, completely different perceived speed.

What the CLI prints when it works

Two commands are worth more than any screenshot.

$ agents-cli info
Extensions (experimental):
  deploy  (override, project) <- deploy-budget-gate: bash scripts/deploy-gate.sh  [requires >=1.5,<2]

That single line is the whole concept. The command, the scope, the extension that owns it, and the exact vector it runs. If you are writing docs for an internal platform team, this is the line to put in the README.

$ agents-cli deploy --help
Usage: agents-cli deploy [OPTIONS] [EXTRA_ARGS]...

  Overrides `deploy` via extension 'deploy-budget-gate' (project scope).

  Runs: bash scripts/deploy-gate.sh

  Argv after the command name passes through to the script verbatim.

  To bypass extension overrides and run the built-in:
  AGENTS_CLI_DISABLE_OVERRIDES=1 agents-cli deploy

The bypass hint is auto generated. Every override ships with an escape hatch, which is the difference between "we shipped a policy" and "we shipped something the on call engineer can get past at 3am".

Then the actual run:

$ agents-cli deploy
running 'deploy' via extension 'deploy-budget-gate' (project scope)
[deploy-budget-gate] projected monthly spend: 612.40 / budget 500
[deploy-budget-gate] BLOCKED: 612.40 exceeds monthly_budget 500

Exit code 1, so CI fails, and the wrapper did it in nine lines of bash plus a budget.json that ops can edit. No amount of policy as code tooling beats a JSON file that a human can change without opening an editor and a PR.

Three rules the schema forced on the generator

If you build something similar, these are the constraints that will bite.

One. run: is not a shell. It is an argv vector. ["bash", "scripts/gate.sh"] is fine. ["bash", "scripts/a.sh && scripts/b.sh"] is not, and neither is anything that looks like a pipeline. Chaining belongs inside the script. The prompt says this in capitals because the first version of a generated extension tried exactly that.

Two. Overrides cannot recurse. The CLI runs your override with AGENTS_CLI_DISABLE_OVERRIDES=1 already set, so exec agents-cli deploy "$@" inside the wrapper hits the built in. Without that, every override of deploy that calls deploy would be a fork bomb. The generated script ends with that exec and it is the single most important line in it.

Three. State is project data, not extension data. A threshold belongs in budget.json at the project root, next to agents-cli-manifest.yaml, because that is what the reviewer will edit. The sandbox therefore seeds any .json, .yaml, .csv or .txt file the extension ships into the throwaway project before the run. Without that, a perfectly good gate refuses on "no budget.json" and the demo looks broken when the design is fine.

Two bugs worth writing down

Reasoning tokens share max_tokens. The first model I tried, deepseek-v41-flash, spent all 6000 completion tokens on internal reasoning and returned content: "" with finish_reason: "length". Twice. The error surfaced as "model did not return JSON", which is true and useless. The fix was to log finish_reason and reasoning_tokens in the error, retry once with double the budget, and default to a lighter model (glm5.3flash) for this task. If you are building against reasoning models, budget for the thinking, not just the answer.

min-width: auto on grid children. The UI shipped with a two column grid, grid-template-columns: 1fr 1fr, which was fine until the right panel contained a <pre> with a long YAML line. Grid items have min-width: auto, meaning the minimum is the content's minimum, so the long line set that column's floor at its own width and squashed the left panel to 128px. It only appeared after compile, because that is the first moment the <pre> had content, which is why the empty state tested clean.

.grid { display: grid; grid-template-columns: minmax(0, 1fr) minmax(0, 1fr); }
.grid > .panel { min-width: 0; }

The fix is one line. The lesson is that any grid holding a code block needs minmax(0, 1fr) and min-width: 0, and that layout bugs need assertions, not eyes. There is now a Playwright check that walks 1440, 1280, 1184, 1024, 921, 920, 820 and 390 pixels, measures both columns, and fails on a ratio outside 0.8 to 1.25 or any horizontal page overflow. It caught the second bug in five seconds.

What I would build next

The most interesting gap is that this tool answers "how do I write this extension", not "what should this project enforce". Point it at a real repo, read agents-cli-manifest.yaml and the Dockerfile, and it could propose the two or three policies that project obviously needs. That is a different product, and a better one.

The second gap is verification depth. An override of deploy currently runs the gate and then hits the built in command, which in a sandbox stops at "no deployment target configured". A dry run mode that fakes the cloud calls would let the demo reach the actual deploy path, which is where the interesting failures live.

The third is CI. npm run compile -- --policy policy.md --out extensions/ with no UI, exiting non zero when the CLI refuses the result, would let a platform team keep policies in git and let the binary gate the PR. That is the version a real org would deploy, and it is maybe 80 lines away.

Try it

python3 -m venv .venv && .venv/bin/pip install google-agents-cli==1.11.0
npm install
cp .env.example .env    # PARTICLE_API_KEY, or any OpenAI compatible endpoint
npm run dev

Then paste a policy, hit compile, watch the CLI argue with the model.

Two commands if you want to see it without the browser:

npm run smoke      # end to end, prints the transcripts
npm run layout     # the grid assertion described above

The extension system is marked experimental, so pin your CLI version. Everything here was built and tested against agents-cli 1.11.0, schema agents-cli-extension/v1alpha1, on macOS with Python 3.12.

The point of the exercise, in one sentence: a model can pick the right command and write the right bash, but it should never be the thing that decides whether its own output is legal.

Code & more: https://www.dailybuild.xyz/project/292-the-override