Ollama 本地大語言模型指南：2026 年按 4GB、8GB、16GB 和 32GB RAM 選擇模型

Originally published on Practical Web Tools.

The most common local-AI question in 2026 is still the simplest one: "I have X GB of RAM, what should I run?" The answer changed a lot in the last year, because newer model generations got dramatically better at small sizes. A 4B model from the current crop beats 7B models from two years ago, which means old "best models" lists are actively misleading.

Here's the current pick per RAM tier, based on actually running these on real hardware, with honest speed expectations for CPU versus GPU. (Choosing between Ollama and llama.cpp as the runtime is a separate question: we compared the two here; this guide assumes either.)

The one rule that decides everything

Model weights at Q4 quantization (the sweet spot for quality per memory) need roughly 0.6-0.7 GB per billion parameters, plus 1-3 GB of overhead for context and the KV cache. Quick mental math for a machine with 16 GB RAM:

• Leave 4+ GB for the OS and browser -> ~12 GB usable
• That's a 14B model at Q4 comfortably, or a 24B at Q4 if you close everything else and keep context short

If you have a GPU, the constraint is VRAM instead. GPU VRAM is worth roughly 3-5x CPU RAM for usefulness, because offloaded layers generate several times faster.

4 GB RAM: small, fast, surprisingly capable

The era of "4 GB can't run LLMs" is over. Current small models handle summarization, rewriting, and simple Q&A genuinely well.

Pick: Qwen3 1.7B for speed, Gemma 3 4B if you can spare the RAM. Avoid anything 7B+; it'll thrash.

8 GB RAM: the sweet spot for capable small models

8 GB machines (base Mac mini territory, older laptops) are the first tier where local AI feels like a real assistant rather than a toy.

Pick: Qwen3 8B or Llama 3.1 8B. Go Gemma 3 12B only if 6+ GB is genuinely free. At Q4 with a browser open, 12B on 8 GB is flirting with swap.

16 GB RAM: where local AI gets serious

This is the tier most mid-range laptops and desktops sit at in 2026, and it lets you run 14B-class models that rival cloud models from a couple of years ago.

Pick: Qwen3 14B at Q4 as the daily driver. On CPU expect roughly 6-10 tokens/sec: reading speed, fine for chat. With a GPU holding most layers, 25-40 tokens/sec.

32 GB RAM: flagship territory

32 GB comfortably runs the models people actually mean when they say "local AI is as good as the cloud now."

Pick: Qwen3 30B-A3B if you want speed, Gemma 3 27B or Qwen3 32B for maximum quality. On CPU, dense 32B is slow (~3-5 tok/s). This tier really wants a GPU with 12+ GB VRAM, where it flies at 20-40 tok/s.

Speed expectations: set them before you install

Honest numbers for Q4 models, single-user chat:

• Modern CPU only: 7-8B -> 10-20 tok/s; 14B -> 6-10 tok/s; 32B -> 3-5 tok/s
• GPU (8 GB VRAM, most layers offloaded): 7-8B -> 40-70 tok/s; 14B -> 25-40 tok/s
• GPU (16+ GB VRAM): 32B fully offloaded -> 20-40 tok/s

Anything over ~15 tokens per second feels instant for chat. Under 5 is where people give up, so on CPU-only machines, cap yourself at 14B.

Why bother running locally at all?

Fair question, when cloud chatbots are one tab away. Two answers. Privacy: a local model's context never leaves your machine, which changes what you can responsibly paste into it. No bank statements or medical records riding to somebody else's server (we wrote up exactly what happens to files you upload to ChatGPT, and the local answer to it). And cost: the runtime price of a local model is your electricity.

Getting started takes five minutes

Install Ollama, run ollama pull qwen3:14b, and you're off. The nicer upgrade is skipping the terminal entirely: a browser chat interface that talks to your local models keeps history on your device and feels like a normal AI product. The private AI chat on Practical Web Tools does exactly that: no account, no uploads, works with locally running models and cloud models alike.

Frequently Asked Questions

Does GPU VRAM count toward these tiers?

Yes: arguably more than system RAM. A model fully held in VRAM runs several times faster than one split across CPU RAM. An 8 GB GPU running an 8B model beats 32 GB of RAM running the same model.

Is Q4 quantization really good enough?

For 8B+ models, Q4_K_M-type quants lose roughly 1-3% versus full precision on most benchmarks, which is imperceptible in chat. Below 4B parameters, quantization hurts more; consider Q6/Q8 for tiny models since they're small anyway.

Can I run a model bigger than my RAM?

Technically yes via mmap streaming, practically no: it's orders of magnitude slower and will hammer your SSD. Stay within the tier tables.

Why is my model so slow even though it fits?

Three usual suspects: context window set very high (the KV cache eats bandwidth), the model split between CPU and GPU awkwardly, or a browser with 40 tabs competing for RAM. Check all three before blaming the model.

Are these model names stable on Ollama's registry?

Tags shift as versions update: qwen3:14b today may point to a newer revision later. Pin exact tags if reproducibility matters.

Which should I install: Ollama or llama.cpp?

For almost everyone reading a RAM-tier guide: Ollama. The differences only matter once you're chasing per-watt performance or exotic hardware: the full comparison is here.

Updated October 2026: gpt-oss changes the 16 GB math

When this guide was written, 16 GB machines were capped at dense 14B models. OpenAI's open-weight release changed that: gpt-oss:20b is a Mixture-of-Experts model (21B total parameters, only ~3.6B active per token) shipped in MXFP4 quantization, so it downloads at roughly 13-14 GB and runs inside 16 GB of memory. That's a model class that used to need 32 GB.

What that means per tier:

• 16 GB (GPU or Mac unified memory): gpt-oss:20b is now the headline pick, with flagship-class answers at usable speed and built-in reasoning that you can dial from low to high.
• 32 GB CPU-only machines: the 20b runs comfortably on pure CPU (some CPU-only setups want the full 32 GB to breathe), which was impossible for a dense 20B+ model.
• 4-8 GB: nothing changes. MoE total size still has to fit in memory, and 13 GB does not. Qwen3 and Gemma 3 small quants remain the right answer.
• gpt-oss:120b (~61-65 GB download) stays Mac Studio / 80 GB+ territory and is not realistic on any laptop.

Two honest trade-offs: MoE speed comes from activating few parameters, so at equal active size a dense model can still edge it on some narrow tasks. But the jump from a 14B dense to a 21B-MoE in quality is unmistakable. And it's a reasoning-first model: it thinks before answering by default, which is superb for hard questions and mildly tedious for "rewrite this email."

Pull it the usual way: ollama run gpt-oss:20b. If you'd rather test the model class in a browser before downloading 13 GB, the site's AI chat connects to local models through Ollama with zero setup.

Updated October 2026: the MoE era changed the value-per-GB math

Three months later, the biggest shift in this space isn't a new dense model: it's that mixture-of-experts (MoE) models now dominate the quality-per-GB charts. A MoE model keeps a large total parameter count but activates only a small slice per token, so you get big-model quality at small-model speed if the whole thing fits in memory.

What that means at each tier:

• 16 GB: gpt-oss:20b (21B total, ~3.6B active, ~13-14 GB in native MXFP4) remains the headline pick: hands-on reports describe 16 GB as the experimentation floor, with 24 GB+ comfortable. On a 16 GB Mac it works but leaves little room for a browser full of tabs.
• 24-32 GB: the standout newcomer value is qwen3-coder:30b (30.5B total, 3.3B active) at roughly 19 GB in Q4_K_M with a 256K context window: capacity of a 30B model, speed of a 3B. Community consensus is that 24 GB is the sweet spot for a genuinely good local coding assistant, and Mac unified memory punches above its weight here. Note the KV cache grows as you fill that giant context, so budget a few GB beyond the weights.
• 64 GB: gpt-oss:120b (117B total, ~5.1B active) runs comfortably in ~65 GB and is the strongest open-weight model most people can self-host today. Earlier reasoning that 64 GB was "overkill" no longer holds.

Two practical notes from the October update:

• Fast RAM matters more than GPU size for MoE. Because only a few billion parameters activate per token, CPU+RAM inference on MoE models feels far closer to GPU speeds than dense-model habits suggest.
• Q4_K_M is still the right default quantization for the 16-32 GB tiers; below Q4 the quality drop on coding tasks becomes visible.

The tier-by-tier picks in the rest of this post still hold; just know that the ceiling of each tier rose: what used to need 32 GB of dense-model memory now often arrives as a MoE that fits in 16-24 GB.

Originally published on Practical Web Tools. Explore all client-side browser utilities at practicalwebtools.com.