跳到正文
DEV Community· Muhammad Abdullah Shah·· 7 小時前AI 評分61

沒有新事實,就不進行第二次修復:面向 AI 智能體的證據優先除錯技能

No Second Fix Attempt Without a New Fact: An Evidence First Debugging Skill for AI Agents

AI 導讀

Muhammad Abdullah Shah 開源了一套供 AI 智能體使用的證據優先除錯技能,規定每次再次嘗試修復前,必須先指出上一輪取得的新事實。作者以一次生產事故説明其背景:DNS 記錄指向私人網絡地址,外部用戶因此無法連線;本地檢查正常,前端和後端日誌對照後,問題在 10 分鐘內修復,事故消耗了每週 token 預算的 6 至 7%。

正文

Every developer working with AI agents knows the loop. Error, guess, error, guess, error. The guesses are fluent and confident and mostly wrong, and each one costs tokens.

I paid for that lesson properly once. A production service was down for an external user while every local check passed. The agent cycled through configuration, ports, firewall rules, restarts. Hours passed. Around 6 to 7 percent of a weekly token budget burned. Then we opened the hosting logs of the frontend and the backend at the same time, and the answer was sitting there in plain text. Fixed in 10 minutes.

The cause was a DNS record pointing at a private network address instead of the host's real public address. Local tests passed because the test machine sat inside that private network. The record was unroutable for the rest of the world. No single log contained this. The truth lived between the DNS surface and the network surface, in the mismatch.

Three lessons came out of that day, and they became a skill.

1. The internet cannot debug your setup

Search engines and model knowledge are trained on other people's systems. Your DNS records, your environment variables, your plan limits exist in exactly one place: your own surfaces. For configuration shaped problems, which are most production problems, your logs are the only judge that knows the case.

2. Retries without new information are waste

The rule the skill enforces is simple: no second fix attempt without a new fact from reality. Before a retry, the agent must name the new fact the last evidence pull produced. If it cannot name one, it does not get to spend. This one gate kills most of the guessing bill.

3. The same failure twice is a fingerprint

One failure is noise. The same failure signature twice, after two genuinely different attempts, means the picture of the problem is stale. The correct move then is not a third guess. It is fresh research, changelogs, status pages, issues opened today, because the case may have been patched this morning and model knowledge is old by default. And if the world is silent, the issue is probably new, so shrink it to a minimal reproduction and work from first principles.

The rest of the discipline

The skill also requires declaring blind spots before starting (which surfaces can you actually reach, and which conclusions stay provisional until you can), evidence pulls at fixed checkpoints, correlation across surfaces before proposing a cause, and converting every understood failure into a regression test. In code, a rule can be a test, and a test enforces itself on every run at zero cost. That is how a loop improves without ever trusting its own memory.

The full skill is open source under MIT: https://github.com/abdullah129026/evidence-first-debugging

It is plain markdown. Copy it into your agent's skills directory or paste it into your project instructions. Real incident reports are the most welcome contribution: if this discipline catches something guessing would not, that story belongs in the repo.

來源:DEV Community · dev.to