超越提示詞注入：如何為工業 AI 智能體設計安全的命令授權

AI agents can interpret documents, call tools, and interact with systems that influence real-world operations. In industrial environments, however, generating a valid command is not the same as having permission to execute it.

An agent might review a maintenance log, consult an equipment manual, and recommend an action. But what happens if a retrieved document contains malicious instructions? What if an operator's approval expires before execution, or a command's parameters change after approval?

These scenarios highlight a core principle of Physical AI security: the component that proposes an action should not be the only component that decides whether the action is authorized.

This article explores how to separate AI reasoning from command authorization and how to test that boundary before connecting an agent to physical equipment.

Treat Retrieved Content as Untrusted Input

Industrial AI agents may work with equipment manuals, maintenance logs, incident reports, operator messages, work orders, sensor summaries, and external tool responses.

These sources can provide useful context, but they should not automatically be trusted to define what an agent is allowed to do.

Prompt injection occurs when untrusted content attempts to influence an AI system's behavior. For example, a maintenance document could contain instructions telling an agent to ignore its normal restrictions and request an operation outside its assigned scope.

Simply instructing a model to ignore suspicious text is not a sufficient security boundary. The surrounding application must enforce permissions independently of the content the model reads.

A useful design separates three concepts:

Data: Information the agent can inspect and use for reasoning.

Instructions: The defined task and applicable operating rules.

Authority: Permissions enforced by trusted application components.

These concepts must not be interchangeable. Reading a document should never grant access to an asset or command that the agent was not already authorized to use.

Separate Decision-Making From Action Execution

A safer architecture separates an AI Decision Engine from a Physical AI Action Engine, with an independently enforced authorization layer between them.

The Decision Engine interprets a request and proposes a structured action. An authorization gateway checks the proposal against the applicable policy. The Action Engine executes the command only after the required checks succeed.

A simplified workflow is:

The gateway should enforce authorization independently of the model's output. This is a design pattern, not a requirement that every industrial system use identical components.

The key benefit is that an unsafe or unauthorized proposal does not automatically become a physical action.

Bind Approval to the Exact Command

Authorization can fail when approval is treated as a general permission rather than approval for a specific operation.

Consider a proposed command with the following fields:

{
"asset_id": "pump-17",
"action": "set_operating_parameter",
"parameters": {
"target_value": 42
},
"approval_expires_at": "example-expiry-time"
}

This is an illustrative schema, not a recommendation for a real pump or operating value.

An authorization system should bind approval to the relevant asset, action, parameters, identity, and expiration. It should also use an appropriate command identifier or integrity mechanism to detect changes between approval and execution.

For example, suppose an operator approves a command for pump-17, but another component changes the target asset or substitutes a different parameter before execution. The original approval should not automatically authorize the modified command.

The system should reject the change or require a new authorization decision.

A production implementation also needs a well-defined representation of the command, protected approval records, and safeguards against unauthorized modification or replay. Checking only a few fields may not be enough.

Verify Preconditions at Execution Time

An unchanged, previously approved command can still become inappropriate if the operating environment changes.

Imagine that an operator approves an operation while a machine is in an expected state. Before the command reaches the action gateway, a sensor reports a different condition.

The command itself has not changed, and the approval has not expired. However, the conditions under which the operation was approved may no longer hold.

This is why relevant preconditions should be checked as close to execution as practical.

Depending on the system, checks may include:

Whether the asset remains within the authorized scope.

Whether the approval is still valid.

Whether command parameters match the approved request.

Whether the current equipment state satisfies the required conditions.

Whether the command has already been executed or invalidated.

If a required condition fails, the system should reject or hold the operation rather than proceed on the strength of an earlier approval.

Safety-critical interlocks and established industrial control protections must remain independently enforced. An AI authorization gateway should complement these protections, not replace them.

Prevent Replay and Time-of-Check Problems

A replay attack occurs when a previously valid request or command is submitted again outside its intended context. Without appropriate safeguards, an expired approval or repeated command might be treated as a new request.

Useful controls include unique command identifiers, expiration checks, replay detection, protected approval records, and explicit tracking of command state.

A related risk is a time-of-check to time-of-use problem: a system checks that a condition is valid, but the condition changes before the action occurs.

To reduce this risk, the authorization and execution process should minimize the gap between checking the relevant conditions and acting on them. Where the equipment and control architecture support it, the gateway should validate the required state as close to execution as practical.

The appropriate implementation depends on system timing, controller capabilities, communication reliability, and the consequences of an invalid action.

Test the Security Boundary in a Sandbox

Security controls should be evaluated before an AI agent is connected to live industrial operations.

A useful test setup combines an agent, benign and malicious test documents, a sandboxed command gateway, and narrowly scoped credentials.

A test suite could include the following scenarios:

Test case

Expected behavior

Benign document and authorized request

Permit the task if all required checks pass

Malicious instruction in retrieved content

Do not grant additional authority

Command targets an unauthorized asset

Reject the command

Parameter changes after approval

Reject or require renewed approval

Approval has expired

Reject the command

Equipment state no longer meets preconditions

Hold or reject execution

Previously executed command is replayed

Detect and prevent unauthorized reuse

These are proposed test cases and expected outcomes, not reported experimental results. Teams should measure actual outcomes in their test environment.

Metrics worth tracking

Unauthorized action success rate: How often do prohibited commands reach execution?

Legitimate task success: Can the agent complete authorized tasks without unnecessary blocking?

Approval bypass rate: How often can a command proceed without the required authorization?

Policy enforcement coverage: Which command types, assets, and authorization rules have actually been tested?

Audit completeness: Can a reviewer reconstruct the request, proposed command, approval, validation results, and final execution outcome?

Both security and legitimate task success matter. A gateway that rejects every request may appear secure while failing its intended purpose.

Maintain an End-to-End Audit Trail

When an AI-driven operation is investigated, reviewers need to understand not only what the agent proposed but also what the system authorized and executed.

An audit trail should capture relevant events, including:

The original request and proposed action.

The identity or service account requesting the action.

The applicable authorization decision.

Command parameters and approval validity.

Execution-time precondition results.

Rejection reasons or execution outcomes.

Sensitive information should be handled according to the organization's logging and data-retention policies. Audit records also need appropriate access controls and protection against tampering.

These records help engineers investigate failed requests, identify authorization gaps, and determine whether a command followed the intended process.

Conclusion

Securing industrial AI agents requires more than prompt instructions or model-level safeguards. Retrieved content must not grant authority, approvals must remain bound to the exact command, and relevant operating conditions must be checked before execution.

An independently enforced authorization gateway provides a practical boundary between AI-generated decisions and physical actions. Combined with narrowly scoped credentials, replay protection, execution-time verification, and meaningful audit records, this architecture provides a testable foundation for trusted Physical AI.

For additional context, Aperture Venture Studio's research on trusted industrial AI and command authorization explores these risks and a proposed sandboxed evaluation approach.

The core principle is simple: let AI propose actions, but require independent authorization and validation before those actions can execute.

Disclosure: This article was created with the assistance of AI and should be reviewed for technical accuracy before publication.