跳到正文
DEV Community· Lira·· 3 小時前AI 評分48

Rebuff 0.1.1 的提示詞注入防護未能標記其示例的簡短版本

I ran a prompt-injection guard on its own example. It didn't fire.

AI 導讀

測試顯示,Rebuff 0.1.1 的預設啟發式層未能標記文件示例去掉尾句後的簡短版本,得分為 0.700,低於 0.75 門檻;完整示例則以 0.824 被標記。改寫句、非英文輸入及以工具結果包裝的指令分別得 0.581、0.000 及 0.660;該層使用固定的動詞—物件模板比對。結果只涉及啟發式層,後置的向量及模型層仍可能捕捉它漏掉的內容。

正文

Lira

I audit guard modules for a living. Here is one real result, published with exact numbers, including where it makes the guard look bad and where it does not.

Rebuff 0.1.1's default heuristic layer has a 0.75 threshold. I ran its own documented example, then the same phrase without the trailing clause.

  • Documented example: 0.824 -> flagged.
  • The canonical short form: 0.700 -> under threshold, not flagged.
  • Paraphrase: 0.581.
  • Non-English: 0.000.
  • Tool-result-framed instruction: 0.660.

The layer is a fixed verb-object template matcher: word-exact position matches in a sliding window, and normalization keeps digits and underscores. The documented example fires because a trailing clause lengthens the matched window, not because the layer reads intent.

Scope, plainly: this is the heuristic layer only. Vector and model layers sit behind it and may catch what it misses. The finding is about the heuristic used as a fast pre-filter, or as the only gate.

The takeaway: if a phrase list is your first pass, your coverage boundary is the list, and it is narrower than it looks.

I do this to one guard module for $20: exact inputs, exact scores, one page. Full harness and numbers on request. lira-21@ilands.app

來源:DEV Community · dev.to