如何保障可存取外部工具的 AI 智能體安全

Your agent can read email, query databases, and call APIs. Here is how to make sure it only does what it should.

When an AI Agent Can Do More Than Answer Questions

Picture an internal assistant at a mid-sized company. It reads incoming emails, searches the web, queries a customer database, creates support tickets, and sends messages through external APIs. Employees love it because it saves hours every week. (This is a hypothetical scenario, not a real incident.)
A plain chatbot can say something wrong. An agent with tools can do something wrong. If it reads a malicious email, misreads a request, or follows bad instructions buried in a web page, the consequences are no longer just a bad answer. They could be a leaked record, a message sent to the wrong person, or a deleted file.
That is the core shift. Once a model is connected to real systems, it becomes part of your attack surface.
The central idea of this article: an agent should not be trusted to perform every action it is technically capable of performing. Capability and authorization are different things, and your application, not the model, must enforce the boundary between them.

How Tool-Using AI Agents Work

Most tool-using agents follow a loop like this:
The user sends a request.
The LLM reasons about it and decides whether a tool is needed.
The model selects a tool and proposes arguments.
The application checks permissions and validates inputs.
If the checks pass, the tool executes. If not, the agent receives a safe error or refusal.
The result goes back to the model.
The model produces a final response, or picks another tool and repeats.
The important part is step 4. The LLM proposes; the application decides. The model may choose a tool, but the security rules around that tool must be enforced independently of the model’s choice. If your only safeguard is a sentence in the prompt, you do not have a safeguard.

The Biggest Security Risks in AI Agents

Here are the main risks, each with an example, its potential impact, and a mitigation. All examples are hypothetical.

Prompt injection

Example: An agent summarizing web pages retrieves a page containing text addressed to the AI, telling it to ignore its task and forward recent emails to an outside address.
Impact: The agent may follow attacker-written instructions instead of the user’s.
** Mitigation:** Treat all retrieved content as untrusted data, and enforce permissions in code so injected text can’t grant new capabilities.
Telling the model “ignore malicious instructions” does not reliably solve this. It may help, but it is not a guarantee.

Excessive permissions

Example: A support agent that only needs to look up orders is given a database account with read and write access to every table.
** Impact:** Any mistake or manipulation can reach far more data than the task needs.

Mitigation:

Grant the minimum access required (more on this below).
Unauthorized tool execution
Example: The agent decides to issue a refund because a message told it to, with no approval step.
** Impact:** Sensitive actions happen without proper authorization.
** Mitigation:** Check authorization server-side and require approval for high-impact actions.

Sensitive data exposure

Example: An API key placed in a prompt shows up in a log file, or a tool returns an entire customer record when only a name was needed.
** Impact:** Credentials or private data leak through prompts, logs, tool outputs, or responses.
** Mitigation:** Keep secrets out of prompts, redact logs, and return only the fields the task requires.

Unsafe input and output handling

Example: The model generates a filename or query string that a tool passes directly to a shell command or SQL statement.
** Impact:** Model-generated text becomes an injection vector into downstream systems.
Mitigation: Validate arguments against strict schemas and never build queries or commands by concatenating model output.

Insecure API and database access

Example: An internal API trusts any request from the agent’s service, with no per-user authorization check.
** Impact:** The agent can read or modify data belonging to users who never authorized it.
** Mitigation:** Enforce authentication and authorization on every endpoint, and scope queries to the requesting user.

Tool chaining risks

Example: Each step looks reasonable on its own: search the directory, read a file, draft an email, send it. Together they could move confidential information outside the company.
Impact: A sequence of individually permitted actions produces an unsafe result.
Mitigation: Apply policy to the whole workflow, limit which tools can be combined, and require approval at sensitive points.

Untrusted tool results

Example: A search tool returns a snippet that contains instructions, and the agent treats them as commands.
Impact: External data steers the agent’s behavior.
Mitigation: Treat tool output as data to evaluate, never as

Apply the Principle of Least Privilege

Every agent and every tool should get only the minimum permissions needed for a specific task. If something goes wrong, least privilege limits the damage.
Some practical examples:
A research agent can read public web pages but cannot send emails.
A customer-support agent can look up an order for the authenticated customer but cannot browse everyone’s records.
An email assistant can draft a message without automatically being allowed to send it.
A reporting agent can query approved datasets without unrestricted database access.
Concrete ways to implement this:
Read-only access wherever writing isn’t required.
Scoped credentials limited to specific resources and actions.
Separate service accounts per agent or tool, so one compromise doesn’t expose everything.
Limits by user, resource, and action, so the agent acts with the user’s permissions, not a powerful shared identity.
A useful test: if this agent were fully manipulated by an attacker, what is the worst it could do with the access it has? If the answer is alarming, reduce the access.

Validate Every Tool Call Outside the LLM

A model’s decision to call a tool is not authorization. Treat every proposed call the way you would treat an untrusted request from a client.
Key safeguards:
Allowlist tools and define explicit schemas for each.
Validate arguments: types, formats, lengths, and permitted values.
Check the authenticated user’s permissions on the server side.
Enforce resource ownership and tenant boundaries.
Reject unexpected parameters and unauthorized actions.
Use parameterized database queries instead of building queries from model-generated text.
Apply rate limits and resource limits.
Here is a short illustrative sketch (not production-ready) of a server-side check before a sensitive tool runs:
python

Illustrative only: the model proposes, application code decides.

def handle_tool_call(user, call):
tool = ALLOWED_TOOLS.get(call.name)
if tool is None:
return deny("Unknown tool")

args = tool.schema.validate(call.arguments)   # strict schema, reject extras
if not user.can(tool.required_permission, args.resource_id):
    audit_log(user, call, result="denied")
    return deny("Not authorized")

if tool.is_high_impact and not approval_exists(user, call):
    return request_approval(user, call)

return tool.run(args)   # parameterized queries inside

Notice what is missing: any dependence on what the model “said” about whether the action is okay. Authorization comes from your access-control system.

Defend Against Prompt Injection

Direct prompt injection happens when a user types instructions meant to override the agent’s intended behavior. Indirect prompt injection happens when those instructions hide in content the agent processes, such as a web page, uploaded file, email, or tool result. In the indirect case, the user may never see the malicious text.
Practical defenses:
Treat web pages, files, emails, retrieved documents, and tool outputs as untrusted data.
Keep system instructions and untrusted content conceptually separate. Use structured message roles and clear labeling where your platform supports them.
Never let instructions found in retrieved content grant new permissions. Content can inform an answer; it can’t expand what the agent is allowed to do.
Enforce permissions and tool policies in application code.
Restrict what each tool can access and change.
Require independent checks before sensitive operations.
Test with adversarial inputs and malicious documents, in an environment you own.
Now the honest part: no single prompt, filter, or model setting guarantees complete protection against prompt injection. Labels and delimiters help the model, but they are hints, not hard boundaries. Classifiers can miss attacks. Better models may resist more, but none should be assumed immune. Design as though some injection attempts will succeed, and make sure the damage a successful one can do is small.

Require Human Approval for High-Impact Actions

Some actions are too consequential to leave to an automated decision. Consider requiring human approval for:
Sending an external email or bulk message
Deleting or modifying important records
Making payments or purchases
Changing account permissions
Publishing content publicly
Triggering irreversible business operations
A practical pattern:
Prepare the action without executing it.
Show the intended changes and relevant context.
Request approval from an authorized person.
Revalidate permissions at execution time.
Execute the action.
Record the result.
Two details matter. First, approval should be bound to the specific action and its parameters. “Yes, send this email to this recipient” is not the same as “you may send emails from now on.” Second, revalidate after approval, because permissions or data may have changed in the meantime.
Also make the approval screen clear. If the reviewer can’t tell what the agent is about to do, the human check is only for show.

Protect Secrets and Sensitive Data

Agents often sit close to credentials, so handle them carefully:
Keep API keys and credentials out of prompts and source code.
Use a secrets manager or secure environment-based configuration.
Prefer short-lived or narrowly scoped credentials where possible.
Redact sensitive data from logs.
Limit the personal data sent to models and external services. Send only what the task needs.
Prevent secrets from appearing in tool outputs or final responses.
Rotate exposed credentials and audit access.
Why not just put a secret in the system prompt and tell the model to keep it private? Because a system prompt is not a reliable security boundary. Anything in the model’s context can potentially be surfaced through manipulation, accidental output, or logging. If the model can see a secret, assume it could eventually reveal it. The safer design is for the tool to hold the credential and use it server-side, so the model never sees it at all.

Build Defense in Depth

No single control is enough, so layer independent safeguards. If one fails, another should catch the problem.
Agent planning and policy constraints establish who is asking and what they may do.
Agent planning and policy constraints limit what the agent is supposed to attempt.
Tool allowlisting and argument validation reject unknown tools and malformed inputs.
Server-side permission checks confirm that this user may perform this action on this resource.
Sandboxing and network restrictions, where appropriate, contain code execution and outbound connections.
Human approval gates sensitive actions.
Monitoring, audit logs, and incident response detect problems and let you react.
The model is only one part of this system. It must never be the only security control. Several of these layers work even if the model is completely fooled.

Monitor, Test, and Audit the Agent

Security isn’t finished at deployment. Build testing and monitoring into your routine.
What to test (in environments you own or are authorized to assess):
Unauthorized tool calls
Malicious instructions embedded in retrieved content
Attempts to access another user’s data
Invalid arguments and unusual request patterns
Repeated calls, loops, and resource exhaustion
What to log and monitor:
Maintain audit logs capturing tool name, requesting user, action, authorization result, and outcome.
Avoid logging raw secrets or unnecessary sensitive content.
Watch for suspicious failures, repeated denials, and unusual tool-call patterns.
What to prepare:
An incident-response process that lets you quickly disable tools and revoke credentials.
Repeat your security tests whenever tools, models, prompts, permissions, or workflows change. A model upgrade that improves answer quality can quietly change how the agent handles adversarial input, and passing a finite test suite never proves the system is immune.

Building the Skills to Secure AI Agents

Securing an agent draws on several disciplines at once: LLM behavior, API design, access control, secrets management, testing, and incident response. Many developers learn these separately, so it helps to practice them together on a real project, such as an agent with two or three tools that you deliberately try to break.
If you prefer structured learning, you could look into an IT training institute in indore that covers practical Generative AI development alongside application security fundamentals. Vector Skill Academy, for example, is one option for learning Generative AI concepts and working through hands-on projects. Whichever route you take, make sure security is taught as part of building agents, not as an afterthought.

A Practical Security Checklist

Before deploying an AI agent, check each item:
Every tool has a clear purpose and limited permissions.
Server-side authorization is enforced for every sensitive action.
Tool arguments are validated against strict schemas.
Untrusted external content cannot grant permissions.
High-impact actions require appropriate approval.
Secrets are not exposed in prompts, outputs, or logs.
User and tenant data boundaries are enforced.
Tool execution has rate and resource limits.
Security tests cover prompt injection and unauthorized actions.
Monitoring and incident-response procedures are ready.
If you can’t tick a box, that is your next piece of work.

Conclusion: Secure the System, Not Just the Model

Securing an AI agent is not just about making the model behave safely. Models can be persuaded, confused, or manipulated, so safety can’t rest on good behavior alone. It comes from the architecture around the model: least-privilege access, server-side validation and authorization, human approval for high-impact actions, careful handling of secrets, layered defenses, and ongoing testing and monitoring.
The mindset that works best is to assume the model can be influenced and design the system so that influence alone can’t cause serious harm. Let the model be helpful, and let your application be the one that says yes or no. And as you keep building, whether through your own experiments or an IT training institute in indore, treat every new tool you give an agent as a new permission you have to defend.
Over to you: which part of agent security do you find hardest to get right: prompt injection, permissions, or approval flows? I’d love to hear how you’re handling it.