如何按智能體的每項操作選擇人工審批或監督模式

Decide the approval mode for each action an agent can take. A draft and a wire should not share one setting. Human in the loop means a person approves before the action runs. Human on the loop means the agent acts while a person monitors and can step in, override, or stop it. I would choose from two tests: can you undo the action, and how wide is the damage if it is wrong?

The European Commission’s expert group on AI, in its 2019 ethics guidelines, defines human in the loop as “the capability for human intervention in every decision cycle of the system.” Human on the loop is the capability for human intervention while the system is designed and for “monitoring the system’s operation.” The guidelines note that intervening in every decision is “in many cases neither possible nor desirable.” In deployment, in the loop means a person approves the email, the refund, or the merge. On the loop means the agent proceeds and a person can reverse the decision or shut the agent down.

Approving every action works for a pilot. It breaks when agents act at volume and in parallel, because the person becomes the bottleneck and then a rubber stamp. Article 14 of the EU AI Act requires that the people overseeing a high-risk system be able “to remain aware of the possible tendency of automatically relying or over-relying on the output,” which the Act calls automation bias. A person who approves 500 actions a day without reading them is technically in the loop and practically absent. The same thing happens in code review when agents write most of the code.

Which actions need a person before they run?

Stay in the loop when a mistake is expensive, the action is hard to undo, the action is regulated, or you have no measured error rate on real cases. A draft can be deleted. A wire transfer or a sent email usually cannot. Approving ten actions a day can be a review. Approving ten thousand is a rubber stamp.

Kris Lovejoy, global head of strategy at Kyndryl, argued in episode 62 that people think about business processes in a linear way and AI does not. She treats the design problem as setting policy constraints so humans do not become a roadblock. She still wants humans in supervisory roles, and she says she is a big fan of agentic AI requiring secondary evaluations and approvals before it can actually complete.

And so I keep pushing back and asking, is it really human in the loop or is it human on top, right?

Kris Lovejoy, global head of strategy at Kyndryl, on Chain of Thought ep 62

I would put that secondary approval on the action that commits something you cannot undo, and let the lookups around it run.

Tyler Akidau, CTO of Redpanda, described a wealth-management demo from a research paper his team wrote. The approval rule sits outside the agent.

But any trade that has more impact than, say, $1,000 on my portfolio always needs to be reviewed by a human. And there is no way for the agent to get past that.

Tyler Akidau, CTO of Redpanda, on Chain of Thought ep 60

The agent makes recommendations. It does not know the rule is there. Infrastructure routes anything over the limit to a person. I would start with that shape for any ceiling you care about: the model proposes, and code outside the model forces review.

Jeetu Patel, Cisco’s president and chief product officer, said in episode 67 that he would consider taking a human out of the loop only after seeing enough decisions statistically to trust them. He expects that point at different times for simple and more complicated use cases. Moving one action from in the loop to on the loop is a smaller step, and it needs the same kind of evidence. That evidence is an error rate on real cases. Without it, a team is guessing.

Which actions can a person review afterward?

Routine, reversible, low-value actions can run on the loop. On the loop is safe only if someone reviews the traces, alerts fire on the right signals, and you can stop or reverse the system. Without those, it is no human at all.

I would start each new action in the loop. Move that one action on the loop after you have a measured error rate on real cases, and only if a mistake is cheap and reversible. Leave expensive, irreversible, and regulated actions in the loop.

How do you score reversibility and blast radius?

Blast radius is what a mistake costs and how far it spreads: money, customers, safety, or legal exposure. Reversibility is whether you can undo the action. One row per action. This table is my sketch of the rule with example classifications; score your own actions.

High cost, irreversible, or unproven: keep a person in the loop. Low cost, reversible, and proven: move it on the loop, with monitoring, alerts, and a way to stop it.

Policy as code is how a row becomes a rule a machine runs: what the agent can access, which actions it can take, and what triggers a halt. The idea came out of cloud, with a pitch that the right rules would end security problems. That pitch did not survive production. Policy as code settled into one important element, beside other controls. It also mismatches agents. It applies deterministic rules to systems that behave probabilistically and run work in parallel, while the rules humans write often assume a linear, step-by-step process. Keep the ceiling in infrastructure the agent cannot see or edit. The $1,000 trade limit is that pattern.

What should the gate record either way?

An audit trail is the record of what the agent did and on what basis: each decision, the context, the tools and arguments, and the output, kept long enough to reconstruct an incident weeks later. You debug failures from it, and you show that the system stayed within policy. It is also what lets you widen scope later, because you can see what was done with the extra room. Observability is the live trace. The audit trail is the retained record built from those traces.

Log the auto path and the human path. A denial belongs in the log before anything runs.

# Sketch only. Not a production gate, and not a control a guest or company runs.

def gate(action, risk, audit):
    record = {
        "action": action.name,
        "args": action.args,
        "reversible": risk.reversible,
        "blast": risk.blast,
        "proven": risk.proven,
    }
    on_loop = risk.reversible and risk.blast == "low" and risk.proven
    record["mode"] = "on_the_loop" if on_loop else "in_the_loop"
    if not on_loop:
        ok = human_approve(action)
        record["decision"] = "approved" if ok else "denied"
        audit.write(record)
        if not ok:
            return None
    result = action.run()
    record["output"] = result
    record["decision"] = record.get("decision", "auto")
    audit.write(record)
    return result

human_approve stands in for a queue you already have. Read risk from the table, in code the agent cannot change. Auto-run, approval, and denial all call audit.write.

Checklist

• Score each action on reversibility, blast radius, and a measured error rate on real cases.
• Keep a person in the loop when the action is expensive, irreversible, regulated, or unproven.
• Move one action on the loop only when it is reversible, low blast, and measured, and keep a stop.
• Enforce the ceiling outside the agent, where a prompt cannot change it.
• Write an audit record for auto-run, approval, and denial, including arguments and output.

The longer explainer, with the episode clips, is on Chain of Thought. It draws on this episode.

Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.

Drafted with AI assistance from the episode transcripts.