本地 LLM 在 Mac 上需要多少記憶體？MacBook M5 16 GB 實測與 16–128 GB 配置表

"Will this model run on my Mac?" usually gets answered with the model's file size. That's only a third of the answer. While a model runs, memory goes to three things: the weights, the context (KV cache) and macOS with everything else you have open. I measured three models on my MacBook M5 with 16 GB, calculated the context cost for classic 8B–70B models, and put together what actually fits into 16 to 128 GB.

Where the memory goes: weights, context, system

Weights. At 4-bit (Q4, MLX 4-bit), about 0.5 GB per billion parameters. 8B is ~5 GB, 14B is 8–9 GB, 32B is 18–20 GB, 70B is ~40 GB.
Context (KV cache). For every token in the context window the model keeps intermediate results on every layer. It grows linearly with length. The catch: Ollama and LM Studio reserve memory for the whole configured window up front, not for the text you actually send.
System and apps. By default macOS lets the GPU use only part of unified memory. On my 16 GB MacBook Metal's recommendedMaxWorkingSetSize is 12.7 GB, about three quarters. The rest belongs to macOS, the browser and the editor.

So the working rule is: weights + context ≤ ¾ of your Mac's RAM. You can raise the GPU limit with sysctl iogpu.wired_limit_mb, but then apps get less and macOS starts swapping.

My measurements on a MacBook M5, 16 GB

Setup: Apple M5, 16 GB, Ollama 0.31, MLX builds. Each model ran in three modes: a short request (a 150-word business email) with a 4K window; a 4.2K-token document (~10 pages) with an 8K window; an 18K-token document (~45 pages) with a 32K window, asking for five key takeaways. Memory is what ollama ps reports. Answers were capped at 200 tokens.

Model, mode	Memory	Prompt reading	Generation	Total time
Qwen 3.5 0.8B, short	1.08 GB	—	109 tok/s	3 s
Qwen 3.5 0.8B, 10 pages	1.14 GB	2257 tok/s	103 tok/s	4 s
Qwen 3.5 0.8B, 45 pages	1.73 GB	1952 tok/s	86 tok/s	12 s
Qwen 3.5 4B, short	3.43 GB	—	34 tok/s	8 s
Qwen 3.5 4B, 10 pages	3.59 GB	390 tok/s	32 tok/s	18 s
Qwen 3.5 4B, 45 pages	4.81 GB	312 tok/s	27 tok/s	67 s
Gemma 4 12B, short	7.64 GB	—	25 tok/s	14 s
Gemma 4 12B, 10 pages	8.29 GB	102 tok/s	16 tok/s	57 s
Gemma 4 12B, 45 pages	9.21 GB	72 tok/s	9 tok/s	4.8 min

What stands out:

Context is cheap on new models. A 32K window added 1.4 GB to Qwen 4B and 1.6 GB to Gemma 12B, roughly 50 KB per token. Most layers in Qwen 3.5 and Gemma 4 attend to a limited window rather than the whole text, so their cache is several times smaller than in classic models.
Long documents hurt latency more than memory. Qwen 4B answers a short question in 8 seconds and a 45-page document in 67. Gemma 12B needs almost five minutes for the same document. A model three times larger reads four times slower.
On 16 GB, an 8 GB model depends on everything else you run. My first Gemma run on the 10-page document happened while I was working on the same laptop: 169 seconds and 4 tok/s. The rerun on a quiet system: 57 seconds and 16 tok/s.

Context cost for classic models

In classic architectures (Llama 3, Qwen 3 and most 2025 models) every layer caches the whole window. The size follows from the config: 2 × layers × KV heads × head dim × 2 bytes. From the config.json files on Hugging Face, at 16-bit (the default in Ollama and llama.cpp):

Model	Q4 weights	Cache per token	8K	32K	128K
Llama 3.1 8B (32 layers)	~5 GB	128 KB	1 GB	4 GB	16 GB
Qwen3 14B (40 layers)	~9 GB	160 KB	1.25 GB	5 GB	20 GB
Qwen3 32B (64 layers)	~19 GB	256 KB	2 GB	8 GB	32 GB
Llama 3.3 70B (80 layers)	~40 GB	320 KB	2.5 GB	10 GB	40 GB
Qwen 3.5 4B / Gemma 4 12B (my measurement)	3.4–7.6 GB	~50 KB	0.4 GB	1.5 GB	~6 GB

At a full 128K window, a classic model's cache weighs as much as the 4-bit model itself. A 70B model with that window needs 80 GB.

What fits into 16–128 GB

Applying "weights + context ≤ ¾ RAM" with the classic-model cache numbers above (newer models with a lean cache fit with room to spare). This is a calculation, not a measurement on every machine:

Mac RAM	For the model	Short requests, 8K	Documents, 32K	Whole books, 128K
16 GB	~12 GB	up to 14B, tight	8B; Gemma 4 12B (9.2 GB measured)	only new-gen 4B
24 GB	~18 GB	14B comfortably	14B (14 GB)	only new-gen up to 12B
32 GB	~24 GB	32B, tight (21 GB)	14B comfortably	8B (21 GB)
48 GB	~36 GB	32B comfortably	32B (27 GB)	14B (29 GB)
64 GB	~48 GB	70B, tight (43 GB)	32B comfortably	32B only with a compressed cache
96 GB	~72 GB	70B comfortably	70B (50 GB)	32B (51 GB)
128 GB	~96 GB	70B comfortably	70B comfortably	70B (80 GB)

How to squeeze a model into less memory

Set the context window for the task. Memory is reserved for the whole window. If you chat in short messages, a 128K window just holds gigabytes empty. In Ollama it's num_ctx, in LM Studio it's Context Length.
Compress the cache. For GGUF models in Ollama, OLLAMA_KV_CACHE_TYPE=q8_0 (with OLLAMA_FLASH_ATTENTION=1) stores the cache in 8 bits and halves it, with little quality impact. That lets a 32B model with a 128K window fit in 64 GB.
Prefer new-generation models. Qwen 3.5 and Gemma 4 spend ~50 KB per context token in my test, classic Llama 8B spends 128 KB. For long documents on a small Mac, that matters more than parameter count.
Close what you don't need. On 16 GB, a browser with a dozen tabs and Xcode compete for the same memory the model needs.

How much memory to buy

16 GB: an assistant for short texts, translation and drafts on 4–12B models. Documents of dozens of pages work, but you wait minutes.
32 GB: the sensible minimum for document work; 14B with a 32K window fits comfortably.
48–64 GB: 32B models, where local AI stops losing to the cloud on everyday tasks.
96–128 GB: 70B and large MoE models, when data must never leave the machine.

How much memory does your Mac have, and which local model do you actually run every day?

Originally published at klukyanov.ru.

Shorter weekly write-ups (in Russian) — on Telegram.