Ken Mazaika 示範如何以對照測試檢驗 AI 寫作風格指南
Your AI Voice Guide Probably Isn't Doing What You Think (Run the Control Test)
Ken Mazaika 以同一模型、同一份篇幅要求為 150–200 個英文單詞的短文簡報,在同一天比較無指南、約 20 條規則的完整指南及 8 條規則的精簡指南。
My agent handed me a draft that opened with a scene from my own life:
On October 19, I wrote down one rule for every overnight OpenClaw job I run: "work on this overnight" is a request, not a scheduler.
I never wrote that sentence. There was no October 19 writing session, no notebook, no rule I sat down and recorded. The agent invented it, because my voice guide told it to open on a concrete, countable instance of my own work.
The draft passed the guide.
That's a bug in the file, but it isn't the interesting one. The interesting one is that I had no way to know whether the file was doing anything at all — and I'd been running it for a month.
The month of credit
Twenty-odd rules. A paragraph cap. A seven-question sign-off at the end of every draft. Everything I published went through it, and every time a draft came out better than the last one, I credited the guide.
I never ran the same sample with it and without it. Not once.
Without a control, a guide gets credit for everything: a draft improves and the guide did it, a draft gets worse and the topic was hard. That's not a voice system. It's a mood with a filename.
The control test
Hold everything still, change one thing.
- One post: a 663-upvote r/OpenClaw thread, tip three — "work on this overnight doesn't work the way you think."
- One brief: a short essay about that post, 150–200 words, no headers.
- Three runs: no guide, the full 20-odd-rule guide, a trimmed 8-rule guide.
The Reddit post is the subject. The essay is the sample. Same model, same brief, same day.
The result
| Text | Words | Sentences | Mean sentence | sd | Em dashes / 1k |
|---|---|---|---|---|---|
| No guide | 185 | 11 | 16.8 | 5.4 | 10.8 |
| Full guide | 163 | 5 | 32.6 | 6.2 | 0.0 |
| Trimmed guide | 183 | 11 | 16.6 | 5.2 | 5.5 |
| My published posts | — | — | 15.2 | 10.3 | 10.8 |
The no-guide version is genuinely good. It opens on the lesson, uses two em dashes, runs sentences from eight words to thirty-five, and invents nothing.
The full guide produced five sentences averaging 32.6 words, zero em dashes, and a fabricated opening. It isn't sloppy — it's compliant. The self-specificity rule demanded a concrete personal moment and the no-fabrication rule forbade inventing one. The first rule won, because a concrete first sentence is checkable and "don't invent a memory" isn't.
The trimmed guide lands on my published shape: mean 16.6 against a corpus mean of 15.2.
The honest size of the win
The trimmed guide beat no-guide on two things: it opened on the number instead of the lesson, and it stopped sooner. That's the whole delta — about thirty seconds of a reader's attention.
So the defensible claim is narrower than the one I'd been making:
A voice guide is a filter for AI idioms. It is not a transfer function for your voice.
Mine caught the phrases I hate — the "it's worth noting," the forced triads, the recap conclusion. That's a real job. It just isn't the job I thought I was paying for.
The eight-rule version I kept
What survived is small enough to read in one sitting, which is the point:
Minimal voice guide
The five spine rules
1. Open on the concrete thing. The first sentence names the tool, number, file, or result. No framing,
no backstory, no "in today's…".
2. One idea per section. If a later passage repeats an earlier beat, cut the later one.
3. Receipt, then verdict. Name the artifact or number, then one line on what it means. A stat never ends
a paragraph.
4. Operator-active first person. "I ran it, it did X." Not "it returns / it packages."
5. Stop when the lesson is done. No recap, no FAQ, no motivational close.
The three voice rules
6. No AI idioms. No "not X but Y" / "it's not just X, it's Y", no forced triads, no em-dash pileups. An
em dash only when it replaces a comma or colon and makes the sentence clearer; never as a rhythm tic.
7. One owned opinion, stated plainly. No both-sides mush. One inline caveat max, only where a claim
needs it.
8. Let sentence length follow the thought. Short punch, then a longer explanation. Don't perform
burstiness, and don't use rhythm as a substitute for a point.
The never list
Words: leverage · utilize · delve · seamless · robust · holistic · transformative · game-changer ·
revolutionize · unlock · "in today's…" · "it's important to note" · "at the end of the day" · "dive in" ·
"in conclusion".
Moves: "not X but Y" / "it's not just X, it's Y" · forced triads · em-dash pileups · rhetorical-question
filler · recap conclusions.
The three questions
1. Does the first paragraph name the concrete thing?
2. Does every claim have a receipt or an owned opinion?
3. Would I say this out loud to a peer?
Three things got cut, and each one is a lesson:
- The paragraph cap. It was unenforceable next to the length target and the rhythm rule — three constraints arguing over the same sentence.
- The self-specificity language. It bought the invented opening. "Be concrete" and "don't fabricate" are not compatible instructions unless you say which one wins.
- The em-dash ban became a permit. The old file talked me out of punctuation I use in every post I actually like.
Run it on your own guide
The kit is a blank template, one Reddit post, one prompt that writes the essay three ways, and the worked example. Clone it, fill the template with the rules you already use, run the prompt, and read the three essays side by side.
- Kit: https://gist.github.com/kenmazaika/c74400587ea6579ab8f20c41e5927394
- The guide that failed, in full: https://gist.github.com/kenmazaika/f6c1e5c5f351668fe7d54b9bad692aea
If your guide can't beat the no-guide version, its rules haven't earned their place. Now you know that in about twenty minutes instead of a month.
What I'd tell you if you only read one line
Test the file before you trust the file. A voice guide that has never been run against its own absence is a preference you're calling a system — and the cost is invisible, because it's paid in your cadence.
Originally published at engineering.kenmazaika.com/blog/give-your-agent-your-voice.
來源:DEV Community · dev.to