本地大語言模型實際需要多少 VRAM？Qwen3.8 27B 的 24GB 與 32GB 顯卡比較
How much VRAM do you actually need for local LLMs in 2026? (Qwen3.8 27B, 24GB vs 32GB cards)
文章以 Qwen3.8 27B GGUF 為例，列出 4K 上下文下各量化約需 14.3–30.7 GB VRAM，並説明實際需求還包括 KV cache 和緩衝區。
Every time a new open model drops, the same question comes back: will it fit on my GPU? Here is how to think about it, with concrete numbers.
1. Memory = weights + KV cache + a little buffer
The GGUF file size is not your VRAM requirement. On top of the weights you need the KV cache, which grows with context length, plus some activation buffer. That is why the same model can fit at 4K context and fail at 32K.
2. Worked example: Qwen3.8 27B (GGUF)
| Quant | Approx. VRAM (4K ctx) |
|---|---|
| Q3_K_M | ~14.3 GB |
| Q4_K_M | ~18.5 GB (~20.4 GB at 32K) |
| Q5_K_M | ~22.2 GB |
| Q6_K | ~25.2 GB |
| Q8_0 | ~30.7 GB |
So on a 24GB card (RTX 3090/4090, RX 7900 XTX), Q4_K_M leaves ~5.5 GB for longer context; Q5_K_M fits but is tight. On a 16GB card, look at Q3_K_M or partial CPU offload. Qwen3.8 uses a hybrid layout where only some layers keep a KV cache, so its long-context cost is far lower than a conventional 27B.
3. 24GB vs 32GB: what the RTX 5090 really buys
- At 4K context, 97 of the 129 models in the quantized.uk index fit a 24GB card comfortably (102 load at all). On the 32GB RTX 5090 it's 103.
- A 70B at Q4_K_M with 8K context is about 47.5 GB, so one 5090 does not change that: you still need two cards or offloading.
- What the extra 8GB buys is context room on 27B–36B models, plus bandwidth: 1,792 GB/s vs 1,008 GB/s on the 4090.
4. Bandwidth decides speed, VRAM decides fit
Cards with the same 24GB run the same models at very different speeds: memory bandwidth ranges from 346 GB/s (Tesla P40) to 1,008 GB/s (RTX 4090). Generation speed roughly scales with bandwidth ÷ weight file size.
5. Macs
A Mac's GPU can only use part of unified memory (about three quarters on a 24GB Mac), so size against that, not the full number.
Tools
I maintain these free, no-signup references:
- VRAM calculator (model + quant + context → weights + KV cache): https://quantized.uk/tools/vram-calc/
- Command generator for llama.cpp / Ollama / vLLM: https://quantized.uk/tools/cli-gen/
- What to run on 8GB / 24GB / RTX 5090: https://quantized.uk/cookbook/
Numbers are estimates plus community measurements; if your real usage differs, tell me and I'll fix it.
來源：DEV Community · dev.to