跳到正文
DEV Community· Omar Baruzzo·· 3 小時前AI 評分62

AI 智能體交接後由誰負責維護?

Who maintains the agent after you hand it over?

AI 導讀

文章指出,AI 智能體交付後能否持續運作,取決於維護責任歸屬,以及接手者是否具備所需資料與工具。作者建議交付維運手冊、由客戶持有的評測集、模型/提示詞/索引的固定版本及書面回退步驟、具名告警負責人,以及不依賴供應商即可讀取的決策記錄。若客戶持有評測集和固定版本,模型棄用通知便可安排測試、比較和切換;否則可能成為落在建置者身上的緊急事件。

正文

TL;DR: ship a runbook, a client-owned eval set, pinned versions with a rollback, a named alert owner and a readable decision log.

An AI agent works on delivery day. What decides whether it still works six months later is who owns it afterwards, and whether that person has what they need. Handover is part of the build, not a meeting at the end.

What changes when nobody touches it

Agents decay without anyone editing them. The model provider retires a version, and the alias you relied on now points at something else. An upstream API renames or drops a field. The documents the agent reads drift away from the ones it was tested on: new templates, new product codes, a different way of writing the same thing. None of this shows up in a code review, because no code changed. It shows up as a mistake in front of a customer, usually noticed first by someone who does not know why it happened.

Five things that leave with the system

The first is a runbook. It says what the agent is allowed to do, what it must never do, which systems it can write to, and what to do when it behaves strangely. It is written for the person on call, not for the developer who built it.

The second is the evaluation set, owned by the client. If the questions and the expected answers live only on the vendor's laptop, the client cannot check anything after a change. With the set in their hands, they can re-run it after a model update, a prompt edit or a new batch of documents, and compare with the last result.

The third is pinned versions with a written rollback. Model, prompt and index are recorded as exact versions, and the way back to the previous combination is a documented step, not a memory.

The fourth is a named recipient for alerts, and it is not the person who built the system. An alert that goes to a mailbox nobody reads is the same as no alert.

The fifth is the decision log, with a way to read it that does not depend on the supplier. When someone asks why the agent did something in March, the answer should not require a phone call to the vendor.

Who gets the first message

Consider the first model deprecation notice. If the eval set and the pinned versions are in the client's hands, it is a calendar item: schedule a test, compare, switch. If they are not, it is an emergency that lands on whoever built the system, at a time they did not choose. That is also a cost problem for the supplier, because unlimited informal support is rarely in the price.

Maintenance is a design constraint

A support contract added after delivery cannot fix a system that was built to be understood only by its authors. The constraint has to be there from the first sprint: what will the client be able to run, check and change without us? Designing for that makes the build slightly slower and the following years considerably calmer.

When your agent breaks in a year, who gets the first message?

Full text: the full handover checklist

Articolo originale: https://www.omarbaruzzo.it/en/blog/agente-ai-dopo-la-consegna

來源:DEV Community · dev.to