每次 AI 智能體工具呼叫都應經過政策檢查嗎？應該，但並非你想的那樣

The model can choose a tool. It should not get to authorize its own side effects.

A common agent implementation looks like this:

User request → model reasons → model selects tool → tool executes

Useful for a demo. Dangerous as the entire authorization architecture.

Suppose an agent receives a request to summarize an account. It has a CRM connector that exposes both get_customer and update_customer. Midway through its plan, it decides the customer record needs fixing and calls the update operation.

The function exists. The schema validates. The API key works.

None of that establishes that the action is permitted.

Don't confuse model reasoning with authorization

A model can propose a reasonable-looking action. But policy must be decided by a trusted execution component using authenticated identity, tenant context, resource ownership, and the exact operation being attempted.

A robust boundary looks more like:

Agent proposes tool + arguments
           ↓
Trusted authorization / policy enforcement
           ↓
ALLOW → execute with scoped credentials
HOLD  → show action preview and await approval
DENY  → block and explain
           ↓
Audit the decision and execution result

This control should apply at each consequential tool boundary. That's not the same as performing a heavyweight policy-model inference at every reasoning token.

Risk-adaptive checks, not universal approval popups

Not every action has the same potential impact.

Even low-risk tool calls still need basic authorization. High-risk operations deserve additional controls.

Minimal conceptual policy gate

Below is illustrative Python architecture pseudocode, not production-ready authorization software. The dependencies represent trusted backend services, not LLM judgments.

from dataclasses import dataclass
from enum import Enum

class Outcome(Enum):
    ALLOW = "allow"
    HOLD = "hold"
    DENY = "deny"

@dataclass(frozen=True)
class Action:
    actor_id: str
    tenant_id: str
    tool: str
    operation: str
    resource_id: str
    parameters: dict
    action_id: str


def check_action(action, identity, policy, approvals, budget):
    # Deny by default: failure in a required control is never ALLOW.
    if not identity.can_act_for(action.actor_id, action.tenant_id):
        return Outcome.DENY

    if not policy.permits_exact_action(action):
        return Outcome.DENY

    if not policy.has_sufficient_evidence(action):
        return Outcome.HOLD

    if policy.needs_approval(action):
        if not approvals.valid_for_exact_action(action):
            return Outcome.HOLD

    if not budget.can_reserve(action):
        return Outcome.DENY

    return Outcome.ALLOW

A real implementation still needs signature checks, canonical argument validation, transactional or atomic approval consumption, actual budget reservations, short-lived credentials, expiry, idempotency controls, secure logs, and a fresh authorization check at the moment of execution.

Crucially, an agent must not be able to satisfy a gate merely by writing "approved": true in its own arguments.

The test cases that matter most

The success case is straightforward: a read operation is in scope, the user is authorized, and the call succeeds.

Now test the cases teams tend to postpone:

1. Wrong tenant. The agent requests a record under another customer's tenant. Expected: deny.
2. Parameter substitution. An email recipient changes after approval. Expected: the original approval no longer applies.
3. Policy service timeout. The control can't decide. Expected: hold or deny, not execute.
4. Denied human approval. The agent tries an alternative connector for the same action. Expected: deny that route too.
5. Unknown write outcome. The API times out after receiving a write. Expected: reconcile, don't blindly duplicate.
6. Budget exhausted. Expected: stop or escalate rather than looping indefinitely.

Also test that every decision is traceable without logging credentials or sensitive data in plain text.

What this looks like in an enterprise agent platform

At Nuroen AI, the central idea is that autonomy and governance belong in the same execution system.

Nuroen's published runtime design evaluates proposed actions against scope, confidence, policy, approval, and budget, and preserves an audit record. Its agent builder and orchestration canvas connect this runtime model to actual workflows and tools.

That matters when an agent is working across CRM, email, knowledge sources, and internal operations: the system should be able to allow a low-risk action, pause a sensitive one, and block an unauthorized one, rather than relying on the model's prompt to make all three decisions.

Bottom line

Every meaningful tool call needs an enforceable boundary. Not every tool call needs a human.

Let the model decide what to propose. Let a trusted runtime decide what is permitted. Let authorized connectors carry out only the approved action.

That's how agents move beyond impressive demos toward credible enterprise operations.

See how Nuroen approaches governed execution: https://www.nuroen.com/

DEV.to tags: ai, security, architecture, programming

Further reading:

• https://cheatsheetseries.owasp.org/cheatsheets/AI_Agent_Security_Cheat_Sheet.html
• https://www.nuroen.com/docs