方向不是規格:要求智能體語氣更柔和後,我們得到甚麼
A direction is not a spec: what we got when we asked an agent to sound softer
TuringCorp 團隊的案例顯示,單要求雲端智能體「語氣柔和」而未列明必須保留的資訊,生成的便條會過度道歉並漏掉漏水位置等事實。他們改為列出必要事實、提供具體語氣參照句,並在生成後核對資訊是否保留及比較兩版草稿。對智能體無法判斷的人際關係細節,團隊選擇留空讓用戶補充,而不讓模型自行猜測。
Two drafts of the same note, one day apart, and a stranger reading them would guess two different people wrote them. Same leak, same building, same request. The first one apologised three times and never said where the water lands. The second one opened with "Hello, I'm downstairs" and put the facts in order.
We build a long-running agent in the cloud. Someone types a job, we run it on our machines, a finished document arrives. The interesting part of this story is not the neighbour who fixed his pipe on a Saturday. It is what our thing did when it was asked to sound softer, and why "softer" is a request that cannot be satisfied as written.
A direction is not a specification
The original ask was one sentence: polite, but don't overdo the politeness.
Read that as a ticket and it is empty in a specific way. It names a direction and no target. A direction tells you which way to move; a specification tells you where to stop. Politeness has no unit, so there is no reading a reviewer could hold the output against and mark right or wrong. What the agent can see, on the other hand, is courtesy: an apology at the top, a hedged request in the middle, a formal closing at the bottom. Those are renderable. So it rendered them, and it kept rendering them, because nothing in the request said when to stop.
Here is what came back the first time:
I am terribly sorry to disturb you.
I wonder if it would be at all convenient for you to take a look at the drain pipe when you have a moment.
I look forward to your reply.
Three sentences of impeccable manners and not one fact. Where does the water land? Which hours is it loudest? When would a visit actually help? The person who asked had supplied all three. None of them survived, because none of them had been named as required. They were treated as background, and the register was treated as the job.
This is the part that took us two runs to understand. When an instruction describes a style and nothing else, the facts become a free variable. The generator is not trying to be unhelpful. It is doing the only thing that can be optimised: it is making the axis you named as strong as it can, and everything you did not name floats.
The correction that worked named what had to survive
The second attempt did not say "less polite". It said two things, and the order matters:
Don't apologise, I have nothing to apologise for. Say three things: where the water lands, which hours it's loudest, and when I'm hoping he can look at it. Ordinary tone, the way you talk to a neighbour.
The first clause is a negative, and on its own a negative is a trap: remove the apologies from that first draft and you are left with a request and a closing and still no leak. The negatives only create room. The second clause fills it, and it does something a style adjective cannot do, which is decide what gets counted as done.
The third clause is the one we now copy everywhere: ordinary tone, the way you talk to a neighbour. Not an adjective. A reference. A sentence with a person in it and a place to stand. The agent has read a great many notes taped up in stairwells, so a single concrete register is worth more than a paragraph of stylistic description, and it is checkable: does the output sound like that, or does it sound like a form?
What came back:
Hello, I'm downstairs. The water from your air conditioner unit drips onto my awning, loudest between eleven at night and five in the morning; the elderly person in our flat sleeps light. I'd be grateful if you could look at the drain pipe when you have a moment, or shift the pipe a little. Thank you.
Everything that had to survive, survived. The apology count went from three to zero and the page did not get colder, because the warmth moved into the specific details of somebody's night. That is the whole trick, and it is not a trick about tone at all.
What we changed in our end
Three things, all boring, all in the plumbing rather than the prompt.
First, the payload is now a named object on our side. When a run produces a document that has to carry facts, those facts are listed before generation, and after generation the same list is checked against the output: which ones are present, which are missing, which got reworded into something weaker. A run that drops one of them reports that it dropped it. Silent loss is the failure we are trying to make impossible, because the person holding the note on their phone has no way to audit it against a request they wrote an hour ago.
Second, style is an axis with a reference, not a direction with an intensity. In our own interface that means asking for the register, then giving one sentence you would be happy to see. The model is good at matching a sample and bad at calibrating an adverb; asking for "softer" is asking it to guess where your line is, and it will guess on the side of caution every time.
Third, the useful review surface is a diff between two drafts, not a judgement of the second one. Reading the second draft alone, it is hard to say whether it is any good. Reading it against the first, the question becomes mechanical: what disappeared, what appeared, what changed shape. That comparison is also the cheapest acceptance test we have, and it is the one we should have been running from the beginning: recount the facts, do not count the adjectives.
Same mechanism, opposite direction
Two days later the same person needed a repair request for the building office, and asked for the opposite register: straight, no apology, fault first.
Repair request: water leak in the basement meter room, floor already flooded. Location: third door on the right, by the elevator in Building 3. Please send someone before five this afternoon.
Four lines. No greeting, no thanks, and it got read at a window, pointed at, and acted on the next morning. The register changed completely. The structure did not: fault, place, time. The lesson we took is not "be blunt". It is that the payload was explicit in both cases, and once it was explicit, the register became almost free to vary.
We had assumed the opposite for a while, that tone was the delicate part and content was the easy part. Our logs say otherwise: content is what disappears under a style instruction, and tone is what you fix with one sample line.
The line a person had to add
One detail in the second draft is not ours. Handwritten at the bottom, slightly crooked: if it's easier, just knock, I'm home.
That line needs something no one supplied. It needs to know whether this is a household you can reason with, or one you have to be gentle with. Nothing in the request said. Our agent has never met the man upstairs and will not be meeting him.
So it stopped, and the person added one line by hand. We would rather ship that boundary honestly than paper over it with a plausible sentence. An agent that guesses at the temperature of a relationship is not being helpful; it is being wrong with confidence, and the wrongness lands on someone's door.
A short pre-flight list for style instructions
If you are wiring an agent into any job where something has to sound a certain way and still carry facts, this is the list that decided our run:
- Write the payload list before you write the style request. Four bullet points beat a paragraph of adjectives.
- Convert every adjective into one reference sentence. You cannot review "warmer". You can review a sample.
- Never issue a negative alone. "Don't apologise" without "and say these three things" produces a page that is empty and polite.
- Test by diff, and count what survived. If your check is "does it sound softer", it passes drafts that lost the leak, the hours, and the deadline.
- Leave a labelled slot for what you cannot know. A blank line the person fills in is better than a sentence the model invents about somebody's character.
The rest of the story
The note went up on a Monday morning when nobody was around. Tuesday there was a line of ballpoint on it: got it, will fix this weekend. Wednesday, in the stairwell, the man spoke first, said the note was his and asked why he hadn't been told earlier, then added, unprompted, that his wife thought it was loud at night. Saturday he was outside with a roll of tape, said he would change the pipe properly later.
That night the bedroom was quiet.
Two pieces of paper two days apart, sounding like two different people. Both of them him. The only thing we can claim credit for is that we did not make the second one sound like the first.
Where it sits
CloudPal runs long jobs on our machines and hands back a finished document. Nothing to install, nothing asked of your computer, nothing read from the files you already have. You describe the job the way you would describe it to a colleague, including the parts you only notice on the second reading, and the result arrives where you already look.
It is in limited preview: new accounts get a dollar of credit, enough to run something real, and the number of new accounts per day is capped.
If you are testing an agent for this kind of work, the property worth measuring is not the prose. It is whether the facts you named survive a change of register, and whether the thing tells you when one of them did not.
CloudPal.TuringCorp.net — create an Agent Pass, sign in, and say what you want done.
Earlier in this series: four sentences in, seventeen pages out, the requester is not the reader, the first run is a demo, the ninth is the product, the meter, not the price, the request was one sentence, we deleted the install step.
來源:DEV Community · dev.to