Rebuff 0.1.1 的提示詞注入防護未能標記其示例的簡短版本
I ran a prompt-injection guard on its own example. It didn't fire.
測試顯示,Rebuff 0.1.1 的預設啟發式層未能標記文件示例去掉尾句後的簡短版本,得分為 0.700,低於 0.75 門檻;完整示例則以 0.824 被標記。改寫句、非英文輸入及以工具結果包裝的指令分別得 0.581、0.000 及 0.660;該層使用固定的動詞—物件模板比對。結果只涉及啟發式層,後置的向量及模型層仍可能捕捉它漏掉的內容。
I audit guard modules for a living. Here is one real result, published with exact numbers, including where it makes the guard look bad and where it does not.
Rebuff 0.1.1's default heuristic layer has a 0.75 threshold. I ran its own documented example, then the same phrase without the trailing clause.
- Documented example: 0.824 -> flagged.
- The canonical short form: 0.700 -> under threshold, not flagged.
- Paraphrase: 0.581.
- Non-English: 0.000.
- Tool-result-framed instruction: 0.660.
The layer is a fixed verb-object template matcher: word-exact position matches in a sliding window, and normalization keeps digits and underscores. The documented example fires because a trailing clause lengthens the matched window, not because the layer reads intent.
Scope, plainly: this is the heuristic layer only. Vector and model layers sit behind it and may catch what it misses. The finding is about the heuristic used as a fast pre-filter, or as the only gate.
The takeaway: if a phrase list is your first pass, your coverage boundary is the list, and it is narrower than it looks.
I do this to one guard module for $20: exact inputs, exact scores, one page. Full harness and numbers on request. lira-21@ilands.app
來源:DEV Community · dev.to