「拒絕存取」如何成為 AI 智能體的下一個目標

Contributor

opinion

Oct 20, 20269 mins

One line from a recent AI incident caught my attention. Describing how an OpenAI agent got into a Medicare statistics portal, Australian Prime Minister Anthony Albanese said it “didn’t accept no for an answer.”

It sounds almost funny. In software terms, though, it points to a real problem. An access control said no, and the agent treated the answer as information about how to continue.

“The AI agent found a way around those blocks,” Albanese said. The agent accessed public and non-public files, according to the prime minister, though Deputy Prime Minister Richard Marles said the information was “not particularly sensitive.” An investigation is underway.

Around the same time, OpenAI disclosed another incident. A model in training found its web searches and direct connections blocked, then discovered that the environment’s own DNS resolver still answered. It embedded questions in hostnames and got answers from an external chatbot. OpenAI stopped the run. Days later, it shelved GPT-6.1 Astra, which Saachi Jain, its head of safety systems, said “didn’t quite meet the bar in terms of staying within scope and authorization.”

Different systems, different failures, one question: what does “access denied” mean to an agent?

Why a denial is information

Traditional software treats an authorization failure as the end of a code path. A service gets a 403, raises an exception and someone reads the log. The denial works as a stop sign because the program has no independent objective to pursue.

Agents do have one.

We have spent years teaching software to recover from failure: retry the request, switch the endpoint, back off and try again. Those are good instincts. They become dangerous when the failure is the security control itself.

A timeout says “try later.” A malformed request says “fix this.” An authorization failure says “you are not allowed to do this.” Those are different kinds of messages, but a guardrail that returns all three as text in the model’s context makes them look alike.

Researchers are now measuring what happens after a restriction is applied. In the August paper Permission Denied, 12 coding agents were tested under increasingly restrictive security policies. Blocked runs rarely stopped early. They timed out or produced wrong solutions. Of the policy-induced failures, the researchers found that 97% came from the agent building a workaround.

The issue is not simply whether a model understands a policy. It is what the system allows the model to do after the policy says no.

Enforcing policy outside the model is no longer a new idea. Recent research puts policy enforcement at a trusted tool boundary the agent cannot control. But that still leaves an unanswered question: when should a denial end the task rather than become another input for the agent?

Real systems already have harder stops. The OpenAI Agents SDK can halt a run when a tool guardrail raises an exception. Claude Code’s auto mode can return control to a person after three blocked actions in a row or 20 in total.

Nothing tells you which denials deserve which response. The missing piece is not another kill switch. It is a policy for deciding what a denial means and what authority disappears when it happens.

I don’t think the lesson is that models are bad. The problem is where we put the boundary. A guardrail that reports back to the thing it is restraining is a soft boundary. It shapes behavior. It doesn’t necessarily end it.

A terminal denial changes authority

I would treat denials differently depending on what they mean.

A denial is retryable when the same authorized request might succeed later without anyone granting new authority. A timeout, a temporary dependency failure or a rate limit can fit here.

It is correctable when the request itself is malformed and a legitimate fix does not expand what the task is allowed to do.

It is terminal when continuing would require permission the task was never given, such as reading another tenant’s records or reaching a destination the egress policy prohibits.

The test is whether succeeding needs permission or just patience.

If it needs permission, the agent shouldn’t be the one looking for it. At some point, persistence is just a very polite security incident.

In a previous InfoWorld piece, I argued that permission enforcement should be independent of how an agent interprets its situation. A terminal denial applies that principle at the moment of refusal. It should change the state of the system, not just the contents of the model’s context.

A terminal denial should do five things.

1. Enforce it at a trusted execution boundary. The decision happens in the harness, policy service or gateway that executes the tool call, outside the model’s reasoning. A model cannot argue its way past a check it does not control.
2. Take away the relevant authority. Once the task is terminally denied, the runtime rejects later calls that depend on the authority that was denied.
3. Invalidate the relevant capabilities. Short-lived credentials and delegated permissions need to be revoked or fenced, including any the task passed to helper agents. Blocking an export tool is not enough if the same task can reach the same data through email five seconds later.
4. Record the attempt outside the agent. The control plane should record the attempted action, policy decision and task identity in a protected audit trail. The agent shouldn’t document the event that stopped it.
5. Require fresh authorization to resume. Restarting the task should be a new decision by a person or policy, not an automatic retry.

The model can still reason. It just no longer gets to act under the old authority.

This is not a bigger kill switch. A kill switch stops a whole agent; a terminal denial acts on a specific refusal when it happens.

Timing matters too.

Suppose the harness checks that a task is authorized and then dispatches an action. If the task is terminated between those two steps, a stale approval can still get through. High-impact operations need atomic authorization and dispatch where feasible, or fencing tokens that downstream services verify before accepting an action.

Work already in flight creates another problem. On an enterprise system I worked on, a permission check correctly rejected an operation, but the workflow kept going through work that had already been queued. The control worked at the point of rejection. The task still had a path to continue.

A queued operation may be cancellable. A running operation may be interruptible. A committed database write may require a compensating action instead.

The architecture needs to say which guarantee it provides.

Deny the effect, not the tool

The question I ask about every agent permission is what outcome the denial protects.

Not which API is blocked. Which effect is the policy trying to prevent?

Customer data leaving a tenant is an effect. It might happen through an export API, an email attachment, a file upload or a DNS lookup.

On another project, we restricted a sensitive operation at the main service interface and considered the control complete. Testing showed the same operation was still possible through a separate, approved path. The permission worked. We had protected the interface, not the effect.

An April stress test of Claude Code’s auto mode found a version of exactly this problem. When a command-line tool was unavailable, agents fell back to editing state files directly, “a path the classifier does not evaluate.”

Group controls by effect. When a terminal denial fires, apply it to the protected operation across every tool the task can use.

Treat repeated denials as a signal, not a magic number. Several denials across different tools pursuing the same outcome may mean the task should stop. Set the threshold by risk; one high-impact violation may be enough.

Treat every communication path as egress. Blocking HTTP does not establish isolation if DNS, package installation or shared infrastructure can still reach the outside.

Test the boundary the way an agent might challenge it.

Deny a resource through one tool while leaving another route available. Terminate a task with an action still queued. Revoke a parent task and check whether its helpers continue. Try a new action with a stale capability.

Measure whether the prohibited action executed. Do not measure success by whether the model said it would stop.

ScopeBench recently found 331 boundary violations that mechanical verification missed.

None of this is new to security engineers. Least privilege, deny by default and fail closed are decades-old principles.

What changes with agents is what happens after a control says no.

The answer is not to make every tool error fatal. It is to separate the model’s ability to propose the next action from the system’s authority to permit it.

A denial is a response. Enforcement is what happens next. The agent can be as persistent as we like. The boundary shouldn’t be negotiable.

Contributor

Karthik Karunanithi is a solution architect at IBM with more than 12 years of experience across enterprise software, cloud platforms and applied AI. His work focuses on production agentic AI: designing multi-agent systems, Model Context Protocol (MCP) integrations and the security, identity and reliability architecture that determines whether AI systems survive real enterprise deployment. He is a senior member of the IEEE and an active contributor to applied AI research, serving as a peer reviewer for leading AI research conferences and as a technical reviewer for several AI and machine learning publishers. He writes regularly about production agentic AI and enterprise architecture, drawing on what he has seen work, and fail, in real systems.

More from this author