NeuronDump 可檢視本地 LLM 結構並輸出精簡版 GGUF

If you run models locally with Ollama or llama.cpp, you've probably downloaded a few gigabytes of GGUF and treated it as a black box. I wanted to open that box: see the layers, the attention heads, the vocabulary, the chat template, and then ask a simple question: which parts does this model not actually need, and what breaks if I remove them?

So I built NeuronDump, a single Rust binary (MIT) with a CLI and a local web UI. It reads GGUF files directly, with nothing hard-coded per model.

👉 Repo: https://github.com/aymen94/NeuronDump
👉 Docs & illustrated intro: https://aymen.xyz/NeuronDump/

What it does

Inspect: architecture, layers (including hybrid SSM/attention models), attention heads, the vocabulary decoded and classified, the chat template as a decision tree, and weight statistics.
Graph: the model as a 3D (or 2D) graph you can open from sections down to single attention heads, neuron buckets and MoE experts.
Cut and see the impact: select any parts and instantly see the size saved, what gets disconnected, and whether llama.cpp still loads the file.
Sweep: cut each layer (or head) in turn, rank them by damage, and write the least important ones out as a smaller GGUF plus an Ollama Modelfile.

The model graph

neurondump web --open

Here's qwen3.8:27b in the 3D city view. Every part is coloured by whether you can remove it: safe, risky, or breaks the model.

Click any block to see its size, quantization and importance, expand it into its children, or add it to the cut selection.

Quick tour from the CLI

neurondump list                          # every local model it can find
neurondump inspect llama3.2:1b           # what's inside a model
neurondump vocab llama3.2:1b --search hello
neurondump template llama3.2:1b          # the chat template as a tree

Slimming is a two-step process: analyze first, then apply a plan you can review.

neurondump analyze llama3.2:1b --out plan.json
neurondump slim llama3.2:1b --plan plan.json --only vocab --out small.gguf

The plan is a plain JSON file, so you can inspect or edit it before anything gets written. --only lets you restrict what gets cut (here, just the vocabulary).

Running the cut model next to the original

Build with the trace feature and NeuronDump runs the cut model in memory next to the original. It compares next tokens, KL divergence, generated text, perplexity and the logit lens per layer.

cargo install --path crates/neurondump --features trace

Some things it showed me

DeepSeek-R1 8B is Qwen3 8B with every weight slightly changed. It has the same neurons in the same places, including one neuron in layer 6 that makes the internal state about 33× larger. Cut that layer and both models produce noise.
On llama3.2:1b, cutting the last layer made the model more confident on one prompt, even though, apart from the first two layers, no layer hurts perplexity on real text more.
DeepSeek's chat template emits tool-call markers that aren't in its vocabulary, so each one gets split into 11 ordinary tokens.

Install

Prebuilt Windows and Linux binaries are on the releases page: unzip and run neurondump list.

Or build from source (Rust 1.88+):

git clone https://github.com/aymen94/NeuronDump
cd NeuronDump
cargo install --path crates/neurondump
neurondump list

Run neurondump with no arguments to choose between the interactive CLI and the web UI.

Limits

It's early, and it has been tested mostly on Windows with Ollama models.

Inference is CPU-only, with no KV cache.
Simulation covers the llama, qwen2/3, gemma, phi3 and bert families. MoE and hybrid models get static analysis only for now.
Zeroing heads or neurons keeps the file loadable but doesn't shrink it.

Known gaps are listed in TEST.md. If you try it on a model I haven't, I'd love to hear what you find: https://github.com/aymen94/NeuronDump/issues