跳到正文
DEV Community· Jeff·· 3 小時前AI 評分58

AI 智能體將具體錯誤改成通用訊息,抹去診斷線索

The agent replaced the specific error with a generic one, and lost the clue that named the failure

AI 導讀

文章指出,AI 智能體在統一錯誤處理時,可能把具體錯誤改成通用訊息,令值班人員失去辨認失敗原因所需的線索。建議把穩定的機器可讀錯誤碼與人類可讀訊息分開,並以結構化欄位保留執行編號、帳戶及餘額等上下文。測試亦應核對失敗分支的錯誤碼或識別欄位,避免只確認有否拋出錯誤。

正文

The payment batch used to fail loudly: "Insufficient ledger balance for settlement run #4412 against account acct-88 (120.00 available, 500.00 required)." The agent was asked to tidy up error reporting, so it consolidated the exceptions into one shared envelope and returned, "Processing failed, please retry." The happy path still works, the test still passes, and at first glance the code looks cleaner. But the message that told on-call which run, which account, and which balance was short is gone.

Why a refactor rounds the message down

When a model is told to centralize or unify error handling, it tends to move toward a short, uniform string. Specific detail reads as clutter in a shared helper: a run id, an account, a threshold, a comparison of available against required. A generic envelope feels like good craftsmanship, and the model optimizes for the cleanest-looking common path rather than preserving each cause.

Nothing in a passing test stops it. Most suites assert that an exception was raised on the failure path, not that the exception carries the field that names why. So the string can be flattened to a generic sentence and the suite stays green, because the check answered "did it fail?" and never "did it fail in a way someone can read?"

What ships when the cause field is stripped

  • On-call starts from zero. A generic message forces the responder to attach logs, reproduce, and hunt the run before they even know the failure was deterministic rather than transient.
  • Deterministic failures look retryable. Insufficient funds will never succeed on retry, but "please retry" invites the queue to requeue it, generating pointless work until a human intervenes.
  • Downstream matching breaks. Automation that keyed on the specific code or cause no longer recognizes the failure class, so routing and alerts fall through to a generic bucket.
  • Monitors lose separation. A transient lock, a validation rejection, and a balance shortage all render as the same string, so alert rules that filter by cause stop working and noise climbs.
  • The regression has no test. Since the test only checks that an error raised, the erosion survives review and ships as a cleanup nobody flagged.

The tell is an error string that became shorter after a "refactor," with the specific cause or identifying fields nowhere to be found. Shorter is better only when the removed words were noise; when they carried the diagnosis, the change moved evidence out of the message.

Keep the diagnostic while fixing the code

Centralizing errors is reasonable; deleting the cause is not. The clean structure separates a stable machine code from a human message, and both keep the identifying context:

  • Keep a stable error code separate from the prose. A machine-readable code for insufficient balance survives envelope changes; the human sentence can be cleaned without erasing the class.
  • Carry context as structured fields, not inline text. Run id, account, available, required, and threshold live in a structured error object, so consolidating the message never deletes the data.
  • Assert the failure path, not just that it raised. A test should check the specific code or the presence of the identifying field on the failing branch, so flattening the string fails the suite.
  • Review error strings as part of the diff. A shorter message that drops the cause is an observability regression, not a formatting improvement; flag it back.
  • Treat the error response as part of a contract on API endpoints. Status code and error body are documented; when a code path changes, re-check that the documented shape still matches what the live endpoint returns rather than assuming a 4xx keeps its old meaning. Where the spec is the contract, we keep that check local in Powerduck and re-run the spec against the live endpoint instead of trusting the markdown.

Clean envelope, intact evidence

A shared error envelope removes repetition, which is a real improvement. Removing the field that names what failed removes the evidence a responder needs at 3 a.m. Simplification should flatten the prose, not the diagnosis; keep the machine code and the structured context, test the failure path, and let the generic wrapper wrap a cause it never had to erase.

來源:DEV Community · dev.to