重寫提示詞前,先查看 Agent 編排器日誌
Read the Logs Before You Rewrite the Prompt
作者檢查 Agent 編排器的執行日誌後,把可確定性處理的任務移入 shell script,減少依賴提示詞規則處理工作分派及任務狀態。在 15 個重建的舊看板狀態測試中,編排器修改其他 Agent 任務內容的情況由 10 次降至 0 次,每小時運行成本由 $0.67 降至 $0.34。其後日誌亦協助找出自動分派腳本的停滯原因,作者以檔案修改時間修正狀態快照,並調整未達標任務的退回方式。
For a while I tried to fix my orchestrator by writing to it.
It runs on GPT-6.1 Sol, and it had habits. It wrote notes into items addressed to other agents. It handed work to the expensive executor when the cheap one would have done. Every time I found one, the prompt got another paragraph explaining why not to. Sol's habits are strong enough that a polite instruction does nothing, so the paragraphs were firm, repeated themselves, and gave reasons. In a week its instructions grew from about 1,500 words to 2,100, and the last fix alone added about 400.
What changed my approach was counting. I asked for the runs of the past day: which roles were launched, how often, and what each launch did. The orchestrator had been launched more than twenty times just to receive a finished task and file the next one. None of those launches needed a judgment. A goal's plan was already a list in order, each item with the role that should do it. The orchestrator read the list and filed the next entry, and sometimes filed it for the wrong role.
Anything that can be decided deterministically is more stable in a script than in any model. That sentence is obvious, and I had still been writing prompts around it.
What moved into scripts
The team works on a board of markdown files, and a resident shell script I call the studio launches whichever agent an item is addressed to. Filing a goal's next task went into that script first. The plan already names who does each item, so when the board has nothing in progress, the studio files the next one for that role and marks it done when the task finishes. The orchestrator now sees a goal item only when I should see its result, when it is the last one, or when something went wrong. The lead executor, which plans, stopped being the default destination for anything vague.
The notes in other agents' items took a script and a sentence. Before each orchestrator pass the studio snapshots every task and review addressed to someone else, and after the pass it puts back any body the pass changed and tells the next pass what it undid. The prompt kept one boundary: into an item addressed to another agent, you write nothing, not even a note. That replaced the 400 words. To check it, I rebuilt fifteen real past states of the board in a scratch copy and ran the orchestrator over each. Before the change it wrote into another agent's item in ten of the fifteen. After, in none.
Two smaller jobs went the same way. Agent failures used to reach me as the last line of a log; now the studio reads the error code each CLI already records in its session files and retries only an overloaded server. Scratch files the agents never cleaned up are deleted once nothing on the board names them and nobody has touched them for three days.
The instructions came back down to about 1,750 words: the 400 were gone, and the rest of that week's growth came from other changes. Per hour the team ran, the orchestrator's cost fell from $0.67 to $0.34. I had been thinking of moving it to Claude Opus to cure its habits. With less and less of its old work left to it, I dropped the idea. Two nights later the team ran a goal from start to finish without a correction from me.
The morning the board stopped
The morning after that night, the board stopped moving at 8:07. I noticed eight and a half hours later.
The orchestrator had sometimes made the wrong call. The script made no call at all. The studio launches the orchestrator when an item reaches a status it should handle, and to avoid launching it twice for the same thing, it remembered which item was in which status. It took that snapshot only between passes. During one pass the orchestrator sent a finished task back to the lead executor, the lead executor finished it again, and all of that happened before the next snapshot. The studio saw the same task in the same status it had already handled and launched nothing. A sprint review sat waiting on that task all day.
The log showed this in a few minutes, and the fix was one field: the snapshot now includes the file's modification time, so a task whose file has changed counts as new even when its status reads the same.
The same stall exposed a second problem, and that one came from my own change two days before. Since the orchestrator could no longer write into the task, it had sent the task back by flipping its status, with the reason written in its own review item. To the lead executor the task looked identical to one whose process had died. Three launches in a row it rechecked the numbers it had already checked and never read why the task had come back. Work that falls short now comes back as a new task, with the shortfall written as its second line.
The habits I had tried to fix in the prompt showed up as behavior I had to notice and then count. The bugs in the scripts showed up as a line in a log. The same logs had shown me, two days earlier, which of the orchestrator's work had never needed a judgment in the first place.
來源:DEV Community · dev.to