Amine Affif 為 AI Agent 設定工具時,為何把限制放在提示詞之外
Giving tools to an AI agent: why I set the limit outside the prompt
Amine Affif 在自建 AI Agent 時,把工具限制設於程序啟動時,而非只依賴提示詞,並預先封閉工具清單。他的 Agent 允許 12 種讀取工具,拒絕 15 種寫入工具;另外設有防提示詞注入段落、每次輸出前的強制對抗性檢查,以及發佈前人工驗證。
When I give tools to an agent, I treat it like a third-party dependency: I don't trust it, I contain it.
The rule that matters: the constraint is set when the process starts, not in the prompt. A prompt is a request. The model can ignore it, or be turned around by malicious text slipped into what it reads.
On an agent I built, the tool list is closed in advance: twelve read tools allowed, fifteen write tools refused. It can't write a file or send a message, even if it tries.
Three more protections:
- an anti-injection section in the prompt, because an agent reads text it doesn't control,
- a mandatory adversarial pass before every output,
- human validation before anything is published.
Why this instead of "a good prompt": an agent that can act does what its environment allows, not what it was politely asked. If the action is impossible, there's nothing left to trust.
I care less about the speed these tools bring than about how to contain them: that's what lets you move fast without pulling code into a repo that nobody has read.
Amine Affif, full-stack developer in Paris. amineaffif.com
來源:DEV Community · dev.to