AI 程式碼審查機械人提示過多，會令團隊逐漸忽略評論

A year ago we turned on an AI reviewer for every PR. Last month we turned most of it off. Not because it was wrong — because it was noisy, and noise is how a review tool loses its teeth.

The decay is predictable

Week one: everyone reads every comment. Week three: the bot flags a formatting nit on line 41, a possible null on line 88, and a question about a variable name. Two of three are bikesheds. By week six, you scroll to the bottom to see if it blocked the merge, and skip the rest.

That's the trap. The bot isn't lying to you. It's just running every check it knows on every diff, and most of those checks have low precision. So the signal-to-noise ratio decays until the comments become wallpaper.

Then it catches a real one — and nobody reads it.

Precision beats recall here

For a review bot, recall is the wrong target. You don't want it to find everything; you want the things it does say to be worth stopping for. A bot that is right 80% of the time it speaks gets read. A bot that is right 40% of the time it speaks gets trained into the background.

The lever is turning down low-confidence rules. Style nits belong in the linter, not a PR comment. Speculation about intent belongs in a suggestion, not a blocking thread. The bot should be quiet by default and loud only when it has something mechanical and high-confidence.

Where the high-confidence stuff lives

The checks that survive the cull are the ones with ground truth. Not "this might be unsafe," but "the code returns 200 but the contract says 201" — a fact, checkable against a machine-readable source. Those have near-zero false positives, so when the bot speaks, it always means it.

For API changes that check is contract drift, and it's the kind of thing that's cheap to run locally and impossible for a human to eyeball. We kept that one. The variable-name comments went.

What to do this week

Open your review bot's history and count, per rule, how often a human pushed back or ignored it. The rules with the lowest strike rate get turned off first. You'll end up with a bot that comments on one in five PRs instead of every PR — and when it does, people read it.

A reviewer that nobody reads is worse than no reviewer, because it creates the feeling of coverage. How long has it been since you actually read every comment your AI reviewer left?