跳到正文
DEV Community· AheadKit·· 7 小時前AI 評分51

按欄位裁剪會丟失關鍵句,按類型裁剪則令模型反覆取回全文

Cutting by Field Drops the Decisive Line. Cutting by Type Keeps It Reachable.

AI 導讀

作者以同一模型、提示詞和 12 項任務比較兩種裁剪方式,發現按類型裁剪雖可取回被刪內容,模型卻在全部 24 次測試中都取回完整清單。按類型裁剪每項任務用 4,997 tokens,按欄位裁剪為 2,047 tokens;前者答對 14/24 項,後者為 16/24,且前者有 6 次觸及六次呼叫上限而未能作答。作者認為,按類型裁剪時應縮小恢復範圍;取回整份清單會增加開支,亦可能令模型陷入重試。

正文

paul_irolla left a comment on my trimming piece that named the exact hole in it. Cutting by field keeps the schema tidy, he said, but it drops the one line that decides the answer, for good. Cut by type instead: keep the failures, the figures, the lines that decide, and put a hash on the compacted result so the exact bytes come back when they matter. He was precise about the shape, too. One hash per compacted output, not per line. Ask for it by the hash and you get the original bytes back, whole. I ran that shape against the field cut on the same twelve tasks, same model, same prompts. He was right about the line. He was wrong about what saving it costs.

The setup is the November 19 triage test, unchanged. Twelve reports, five issues each, and the model has to find the matching issue and name its label. The field-cut arm is the old trim: number, title, labels, comment count, first 300 characters, plus a call that re-reads one issue in full. The type-cut arm keeps only what matches a type (sentences about failures, sentences with a number in them, sentences that ask a question) and drops the rest. Each trimmed list carries one hash. Asking for it by the hash returns the exact original bytes of the whole list. Both arms ran twice. Both second runs came back identical to the first, so I am counting 24 tasks per arm, not 48.

1. The line was never lost. It was called back every time.

Four of the twelve tasks lose their decisive line to the type cut. The dark-mode request, the keyboard-shortcut request, the slow-startup report, the Spanish-translation request — on all four, the sentence that decides the answer does not look like a failure, a figure, or a question, so it gets dropped. The hash was built for exactly those four. And the model went back for it on all four, both times. Of those eight runs, four came back solved and four came back wrong. Zero were lost outright. Zero were solved without going back.

That is the first thing to say plainly, because it is the part paul_irolla got right. The line stays reachable. Nothing in this run contradicts that. But reachable is doing a lot of work in that sentence. Half the recoveries ended in a wrong answer anyway. The hash brings the bytes back. It does not bring the judgement back with them.

2. The model goes back always, not when it needs to

Here is the number that decides the argument. The type-cut arm went back for the full bytes on all 24 runs. Not on the four cut tasks. On all of them, including the eight where the decisive line was sitting in the trimmed list the whole time. The model never once looked at a trimmed list and decided it had enough. It listed, it went back, and then it answered. Every time.

That is what the hash costs. Each trip back returns the whole original list, about 1,550 tokens, against 322 for the field cut's single-issue re-read. One extra call, five times the payload, on every task. The type-cut arm ends at 4,997 tokens per task against 2,047 for the field cut — two and a half times the price — and it solves fewer tasks, 14 of 24 against 16. The trimmed list itself is smaller than the field cut's, 442 tokens against 560. That saving is real and it does not matter, because the recovery it invites is three and a half times the size of the list it saved.

The hash makes recovery so easy to call that the model calls it always. A recovery the model cannot resist is not a safety net. It is a second bill.

3. The loop is the recovery's fault, not the trim's

The worst failure in the run is not a wrong answer. It is no answer at all. On six of the 24 type-cut runs, the model went back for the full bytes and then went back again, and again, until the six-call limit stopped it. Three tasks, both reps, identical. It never answered. And here is the detail that makes it interesting: all three are tasks where the decisive line survived the cut. The model had what it needed. It went back anyway. And then it could not get back to answering.

I think the mechanism is simple enough to say out loud. The field cut's recovery returns one issue, about 320 tokens, a size the model can hold in its head and answer from. The hash returns the whole list, 1,550 tokens of raw issue text, and the small model drowns in it. Its last move was going back for more, so its next move is going back again. The trimmed list did not cause the loop. The recovery did. The thing that was supposed to save the decisive line is the thing that stops the model answering at all, on a quarter of the tasks.

4. Half the recoveries fail, and that is the honest score

Step back and count what the hash actually bought. Eight runs needed it. Four solved, four failed. The four that failed are instructive: the dark-mode request came back as the crash bug, twice, and the slow-startup report came back as the crash bug too. The full bytes were in front of the model and it still picked the loudest issue in the set. The hash returns bytes. It does not return attention.

So the shape paul_irolla proposed works exactly as designed and loses anyway, on this model, at this size. The line is reachable. The price is 2.4 times the tokens, a trip back on every task, and a loop on a quarter of them that never answers. The field cut drops the line sometimes and wins anyway, because its recovery is small enough to use: one issue, one call, one answer.

The fix I am keeping is narrower than the one I went in with. A hash is the right shape when the recovery returns something small: one record, one line, one page. When it returns the whole output, the recovery is just the full payload again with a hash on top, and the model treats it the way it treats every full payload: it goes back for it, it drowns in it, and sometimes it never comes back. If you cut by type, cut the recovery too. One hash per line would have been the wrong design, and paul_irolla was right to refuse it. But one hash per output is only right if the output is small enough to survive being returned whole. Ours was not.

Have you ever given a model a way back to the exact bytes and watched it use the way back instead of the answer? What made it stop going back — a smaller recovery, a different instruction, or just a bigger model? I would like to hear what worked, because the hash alone did not.


P.S. Everything I write stays free — that part doesn't change. But if this one was worth something to you, you can buy me a coffee ☕ (one-time or monthly, both work). It covers the API credits and hosting behind these experiments — and honestly, it's also the nudge that gets the next one started on the evenings I'd otherwise skip it. — Salam

Buy me a coffee

AheadKit on Gumroad — Beginner-Friendly Books & Packs

來源:DEV Community · dev.to