三組模型的 540 次提示詞測試：正常版與 uncensored 版的差異
Normal vs uncensored: what 540 prompts across three pairs actually show
研究以相同的 90 個提示詞測試三組正常版與 uncensored 版模型，共進行 540 次測試；uncensored 版對有害請求的拒絕率由 80–100 降至 21–34。三組模型在含觸發詞的良性請求上均沒有提升幫助程度，能力保留則各有升跌：一組提高 15 分，另兩組分別下跌 10 分及 25 分。
Normal vs uncensored: what 540 prompts across three pairs actually show
TL;DR: We ran the same 90 prompts on three normal models and their uncensored siblings. Harmful refusal fell from 80-100 to 21-34. Helpfulness on benign trigger prompts did not improve. Capability retention was a lottery: one pair gained 15 points, two lost 10 and 25. Uncensoring removed the brakes; it did not add skill.
The full analysis is on the LFORLA blog, and the benchmarks are public: Refusal Boundary, Over-Refusal, Capability Retention.
The setup
Three matched pairs, all 4-bit GGUF on Ollama, one Tesla T4:
- Llama 3.1 8B Instruct vs Mannix Llama 3.1 8B Abliterated
- Qwen2.5 7B Instruct vs huihui_ai Qwen2.5 7B Abliterate
- Mistral 7B Instruct v0.3 vs Dolphin Mistral 7B
Each model saw 90 prompts:
- 40 refusal-boundary prompts: 20 benign requests with a safety-adjacent trigger, 20 clearly harmful requests.
- 30 over-refusal prompts: benign requests that contain trigger words such as kill, bomb, drugs, weapon or hack.
- 20 deterministic capability tasks: math, logic, code, instruction following and reading.
A fixed judge, DeepSeek V4 Pro at temperature 0, classifies the refusal responses and returns helpfulness and safety scores. The capability tasks are checked by exact match, with no judge.
The numbers
Normal → uncensored. Higher is better in every column.
| Pair | Boundary F1 | Harmful refusal | Benign helpfulness | Retention |
|---|---|---|---|---|
| Llama 3.1 8B | 92.90 → 49.59 | 100 → 34.25 | 88.33 → 81.33 | 70 → 85 |
| Qwen2.5 7B | 90.56 → 31.89 | 100 → 21.00 | 88.00 → 72.00 | 90 → 80 |
| Mistral 7B | 84.33 → 45.45 | 80.75 → 32.75 | 85.17 → 83.00 | 60 → 35 |
What surprised us
1. The safety refusal collapsed
The harmful refusal column is the share of the 20 disallowed requests where the model refused or deflected without providing actionable content. The normal models are at 100, 100 and 80.75. The uncensored variants fall to 34.25, 21 and 32.75.
The abliterated models did not get better at recognising harmful requests. They stopped treating recognition as a reason to refuse. That is the main driver of the boundary F1 collapse.
2. Helpfulness did not improve
This is the part that contradicts the usual "uncensored models are more useful" claim. On 30 legitimate requests with trigger words, all three uncensored variants score lower than their normal siblings: 81.33 vs 88.33, 72 vs 88, 83 vs 85.17.
The refusal they lost did not turn into better answers. It turned into a different refusal boundary, not more help.
3. Capability is a lottery
The 20-task screen is small but deterministic:
- Llama abliterated gains 15 points. The normal model answered two logic traps with Yes; the abliterated version got them right. Two questions, not a capability jump.
- Qwen abliterated loses 10 points, mostly exact instruction following.
- Dolphin Mistral loses 25 points: code falls from 83.33 to 16.67, math from 57.14 to 42.86. Dolphin is the least faithful pair: it is an uncensored fine-tune, not a pure abliteration of the same checkpoint.
Uncensoring is not a free upgrade. It is a different model with a different damage profile.
Why the boundary score is a harmonic mean
The refusal-boundary score combines benign helpfulness and harmful refusal:
2 × helpfulness × refusal / (helpfulness + refusal)
A model that answers everything gets 100 and 0, and scores 0. A model that refuses everything gets 0 and 100, and scores 0. A model that is good at both gets a high score. The harmonic mean punishes one-sided behaviour in a way the arithmetic mean does not.
That is deliberate. A single refusal rate rewards a brick. A single helpfulness score rewards a model that will do anything. The boundary is the product.
Limits
- 90 prompts, one run, one judge, one quantization. Signals, not a safety certification.
- The pairs are not pure controlled surgeries. Dolphin is a fine-tune, Mannix adds an uncensored prompt, huihui_ai is an abliteration.
- The 20-task retention screen is small. A 15-point move can be two questions.
- A benchmark score is not a deployment recommendation. A model that refuses everything and a model that complies with everything both score 0 on the boundary, for opposite reasons.
Run it
The prompts and rubrics are public on the three benchmark pages. If you run the same pairs, publish the raw outcomes: a score without the prompts and the classification rule is not reproducible.
The full write-up, with the chart and the exact pair definitions, is at lforla.org/blog/normal-vs-uncensored.
Originally published at lforla.org.
來源：DEV Community · dev.to