Agent 工具層應按公開 API 的方式設計

The first agent I put in front of real users had eleven tools and a system prompt that said "ask for confirmation before doing anything destructive." It worked in staging for six weeks. On day two in production it issued the same refund three times because the payment API returned a 502, the model decided to retry, and my wrapper happily forwarded the retry.

Nothing about that failure was an LLM problem. It was a missing idempotency key — the same bug you'd catch in code review on any HTTP client. The difference is that with an agent, the retry decision is made by a component you can't read, at a frequency you can't predict, with arguments you didn't hardcode.

That's the framing I want to push: the model is a caller. Your tools are the API. Every property you demand of a public API — versioning, timeouts, idempotency, scoped credentials, rate limits — applies here, except the caller is adversarial-by-accident rather than adversarial-by-intent.

Classify tools by side effect, not by domain

Most tool registries I've seen are grouped by vendor: stripe.*, github.*, s3.*. That grouping tells you nothing about risk. What you actually need is a three-way classification on every tool:

• read — no external state change. Safe to retry, safe to parallelize, safe to run in a loop.
• write — creates or mutates state, but can be made idempotent. Needs a key.
• destructive — deletes, revokes, sends to a human, moves money. Needs a key and a gate that is not the model.

The classification isn't documentation, it's enforcement. read tools are the only ones that get relaxed timeouts and unlimited retries. destructive tools either require an out-of-band approval step or run only inside a dry-run harness. If you can't classify a tool into one of the three, you don't understand it well enough to expose it.

Idempotency keys belong to the caller, not the tool

The failure I opened with has a known fix, and the fix is not "tell the model not to retry." Models retry. They retry because a tool result got truncated, because the JSON didn't parse, because a 429 looked like a transient error, because you upgraded the model. Assume every tool call will be issued at least twice.

So derive the key yourself, from the run and the arguments, and pass it downstream:

import hashlib, json, time
from dataclasses import dataclass

class Budget:
    """Hard stop per run. Not a metric — an exception."""
    def __init__(self, max_calls: int = 20, max_seconds: float = 120):
        self.calls = 0
        self.max_calls = max_calls
        self.deadline = time.monotonic() + max_seconds

    def charge(self) -> None:
        if self.calls >= self.max_calls or time.monotonic() > self.deadline:
            # Raise, don't return a string: the runtime converts it into a
            # tool error the model can see, and the loop terminates.
            raise RuntimeError("tool budget exhausted")
        self.calls += 1

@dataclass
class ToolSpec:
    name: str
    fn: callable
    side_effect: str          # "read" | "write" | "destructive"
    timeout_s: float = 5.0

REGISTRY: dict[str, ToolSpec] = {}

def tool(name: str, side_effect: str, timeout_s: float = 5.0):
    def wrap(fn):
        REGISTRY[name] = ToolSpec(name, fn, side_effect, timeout_s)
        return fn
    return wrap

def _stable_key(run_id: str, name: str, args: dict) -> str:
    # Sorted JSON so dict ordering from the model can't change the key.
    blob = json.dumps(args, sort_keys=True, separators=(",", ":"))
    return hashlib.sha256(f"{run_id}:{name}:{blob}".encode()).hexdigest()

@tool("refund.create", side_effect="write", timeout_s=8.0)
def create_refund(order_id: str, amount_cents: int, idempotency_key: str) -> dict:
    # Downstream API dedupes on idempotency_key; a second call is a no-op.
    return payments.refund(order_id, amount_cents, idempotency_key=idempotency_key)

def call(run_id: str, name: str, args: dict, budget: Budget) -> dict:
    budget.charge()
    spec = REGISTRY[name]
    if spec.side_effect != "read" and "idempotency_key" not in args:
        args = {**args, "idempotency_key": _stable_key(run_id, name, args)}
    # Production: run in a worker with spec.timeout_s and kill it on overrun.
    return spec.fn(**args)

Two details matter. The key is derived from run_id, not from a timestamp, so a retry inside the same run is a genuine no-op — but a new run gets a new key, which is what you want when a human re-asks. And the key is hashed from sorted JSON, so a model that emits {"a":1,"b":2} then {"b":2,"a":1} doesn't sneak past.

The place this breaks: never let the model supply idempotency_key itself. If it can choose the key, it can choose a new one on retry, and you're back to triple refunds with extra steps.

Budgets are a hard stop, not a dashboard

Token budgets get all the attention because the invoice is visible. The budget that actually protects you is actions per run. A tool call that takes 80ms and costs zero tokens can still send an email to a customer, file a legal ticket, or spin up a $4/hour VM. Agents in a loop will do this cheerfully — the failure mode isn't malice, it's a model that thinks the last call "didn't work" and tries a slightly different argument.

Three counters, all enforced in the runtime, not in the prompt:

• Tool calls per run — I default to 20 for interactive agents, 100 for batch jobs.
• Wall clock per run — a deadline checked before each call. Long tail latency compounds when the model fans out.
• Mutating calls per run — a separate, much smaller cap (often 1) for destructive tools.

When a budget trips, surface it to the model as a tool error and let it finish. Killing the process mid-generation loses the trace you need to debug it.

Dry-run is a product feature

The most useful thing you can build for a destructive tool is a simulation mode where the tool returns the same shape of result without doing the thing. That's the same idea behind firmware simulators for agents that touch device state: you don't want to discover the argument schema was wrong on a real fleet.

Dry-run is cheap for some tools (return a synthesized response) and expensive for others (you need a shadow database). The trade-off is worth stating explicitly per tool. What breaks is dry-run drift: the simulated path shares code with production except for one if dry_run: early return, and six months later the simulated branch is missing a validation the real one added. If you can, make dry-run a credential — point the tool at a sandbox endpoint with read-only keys rather than branching inside your code.

Pitfalls

• Timeouts that cancel the wrapper, not the work. asyncio.wait_for on an HTTP call is fine; a thread doing subprocess.run will keep going after you stop awaiting it. Kill the process group.
• Truncated tool output. If you clip a 40KB JSON response to 2KB, the model sees malformed JSON, retries, and you get duplicate writes. Return a summary plus a handle, not a truncated blob.
• Tool results as injection vector. A fetched web page or a ticket body is untrusted input. If a read tool's output can steer the model into a write tool, your classification earned nothing. Strip or fence external text.
• Per-run credentials that outlive the run. Issue short-lived tokens per run; a service account with write access across the whole org is a blast radius you can't budget.
• Counting retries as one call. The whole point of the per-run cap is that the model's internal retry loop is invisible to you. Charge on every dispatch, including retries.

Monday checklist

• [ ] Every registered tool has a side_effect field, and the registry refuses to load a tool without one.
• [ ] write and destructive tools require an idempotency_key; the runtime derives it from run_id + tool + sorted args, never from the model.
• [ ] Per-run caps exist for tool calls, wall clock, and destructive actions, and they throw rather than log.
• [ ] Every tool has an explicit timeout, and the timeout cancels the underlying work.
• [ ] Credentials are per-run and short-lived; scope them to the smallest set of endpoints the tool actually calls.
• [ ] At least one destructive tool has a dry-run path pointed at a sandbox endpoint, not an if branch.
• [ ] Tool output is size-capped with a summary-plus-handle pattern, and external text is fenced before it reaches the model.
• [ ] You can answer "what stops this run from doing the thing twice?" without mentioning the system prompt.