Inkling 2 Mini 可在 32GB MacBook 離線運行,循環架構為何重要
Inkling 2 Mini Runs on Your MacBook. Here's Why That Matters
Thinking Machines Lab 於 2026 年 11 月 20 日發佈 Inkling 2 Mini,這款 26.4B 參數的循環 Transformer 約佔 20GB 記憶體,可在 32GB MacBook 上離線運行,採用 Apache 2.0 授權。
文章將本機部署門檻與循環架構的計算取捨並置,並以編碼基準和事實回憶數據呈現其較適合的任務範圍。
Inkling 2 Mini Runs on Your MacBook. Here's Why That Matters
Thinking Machines Lab just released Inkling 2 Mini, a 26.4-billion-parameter looped transformer that fits inside 20GB of memory and runs offline on a 32GB MacBook. The model shipped on November 20, 2026 under Apache 2.0, and within hours it was the number-one trending model on Hugging Face. On r/LocalLLaMA, the launch thread hit 1,550 upvotes and 454 comments — the kind of reception that tells you the local-inference community has been waiting for exactly this.
Read the full version with charts and embedded sources on ComputeLeap
The headline numbers are striking: Inkling 2 Mini ties Qwen3.8-27B at 20.1 on the Artificial Analysis Intelligence Index, leads sub-100B open-weight models on DeepSWE v1.1 (48.2%), and scores 79.2% on SWE-bench Verified. It does all of this with just 3.4 billion active parameters per token — less than a tenth of its total weight count. The trick is a looped architecture that runs the model's middle layers three times per token, squeezing more reasoning out of fewer stored weights.
But the real story is not benchmarks. The real story is what happens when a model this capable fits on hardware that millions of developers already own.
What Looping Actually Does
Most transformer models are strictly sequential: a token passes through every layer exactly once, top to bottom. Inkling 2 Mini breaks that pattern. Its 24 layers are organized into four blocks of six. The first and last blocks run once per token, as usual. The two middle blocks — 12 layers — run three times each. This gives each token 48 layer applications from only 24 layers of stored weights.
The mechanics matter. Each loop pass after the first receives the previous pass's output through a learned gate (what Thinking Machines calls "attention residuals"). The model is not simply re-reading the same representation — it is iteratively refining it, compounding reasoning depth without increasing the storage footprint.
The official announcement reports that a non-looped baseline needed 140% more inference compute to match the looped model's loss on 512K-token agentic traces. Put differently: looping bought the equivalent compute gain of roughly doubling the model at a cost of 7.5% more training compute.
This is not the same as chain-of-thought. Chain-of-thought makes the model produce more tokens — visible reasoning that you pay for. Looping makes the model think harder per token without producing additional output. The companion variable-loop research model (also released on Hugging Face) showed that looped models produce shorter chains of thought while maintaining legibility. No increase in covert sabotage behavior appeared on LinuxArena safety tests.
The Architecture Under the Hood
Inkling 2 Mini is a sparse Mixture-of-Experts (MoE) model. Each layer contains 128 experts, of which 8 are selected per token. Training started with 384 experts; two-thirds were pruned mid-training, keeping only the most-used 128 per layer. After pruning, the model was "healed" with 1 trillion tokens distilled from a larger internal Inkling model, followed by supervised fine-tuning and 2.2 trillion tokens of reinforcement learning.
Key specs at a glance:
| Spec | Value |
|---|---|
| Total parameters | 26.4B |
| Active per token | 3.4B (5.4B with looping) |
| Layers | 24 stored, 48 effective |
| Experts per layer | 128 (8 active) |
| Context window | 1M tokens |
| Weight size | ~20GB |
| Precision | NVFP4 + MXFP8 (training = deployment) |
| License | Apache 2.0 |
| Modality | Text + image input, text output |
Note: The 5.4B "effective active" count comes from counting the looped layers' activations three times. The physical memory footprint stays at ~20GB regardless of loop count. Per-token compute, however, scales with loops — expect roughly 1.5-2x the FLOPS of a standard 3.4B-active model.
One detail worth noting: the weights ship in the same 4-bit and 8-bit mixed precision used during training. There is no quantization conversion loss. What you download is what the model was trained with — a rare property that eliminates the usual "how much quality did I lose by quantizing?" question.
Benchmarks: Where It Leads, Where It Trails
Thinking Machines positioned Inkling 2 Mini as the best sub-100B open-weight model for agentic coding tasks. The benchmarks largely support that claim — with important caveats.
Where Inkling 2 Mini leads:
| Benchmark | Inkling 2 Mini | Qwen3.8-27B |
|---|---|---|
| DeepSWE v1.1 | 48.2% | 42.2% |
| SWE-bench Pro v2 Hard | 77.9% | 73.8% |
| SWE-bench Verified | 79.2% | 77.8% |
| Vibe Code Bench v1.1 | 77.9% | 64.9% |
| BrowseComp (compaction) | 79.0% | — |
| DeepSearchQA (compaction) | 85.3% | 79.7% |
| MCP Atlas | 82.7% | 77.3% |
| ARC-AGI v2 | 49.3% | 42.4% |
| ScreenSpot Pro | 76.5% | 57.6% |
Where it trails:
| Benchmark | Inkling 2 Mini | Qwen3.8-27B |
|---|---|---|
| AA-Omniscience | -8.6 | -6.4 |
| Non-hallucination rate | 63.4% | 71.4% |
| AutomationBench v1.0.6 | 41.8% | 48.2% |
| Terminal-Bench v2.1 | 71.6% | 73.0% |
| MultiChallenge | 50.0% | ahead |
The pattern is clear: Inkling 2 Mini excels at deep, single-task agentic work — coding, search, tool use — and stumbles on factual recall, multi-turn conversation, and tasks with many constraints. This aligns with what looping should theoretically do: more reasoning cycles per token improve logical depth but add zero new stored knowledge.
Artificial Analysis reports that Inkling 2 Mini completes Intelligence Index tasks in 9.0 minutes versus Qwen3.8-27B's 38.2 minutes — roughly 4x faster — at comparable scores. The cost per task comes in at $0.59 versus Qwen's higher compute bill. For local inference, that speed advantage translates directly to tokens per second.
Plotly's independent data-analysis benchmark scored it 36 of 43 questions correct (84%), just three fewer than the much larger Inkling 2 Small. Not bad for a model one-twentieth the size.
Contrarian Corner: The benchmark story has a hole. Looping amplifies compute per token without adding stored facts. If your daily workflow is research-heavy — pulling citations, synthesizing across sources, checking claims — Inkling 2 Mini's factual recall (63.4% non-hallucination rate) will bite you. The 20.1 Intelligence Index score ties Qwen3.8-27B, but that index weights agentic coding heavily. For knowledge work, denser models remain the better choice.
Running It Locally: The Setup
This is where the local-inference community gets excited. Inkling 2 Mini's ~20GB footprint means it fits comfortably on:
- Any Mac with 32GB+ unified memory (M3 Pro and up, M4 Pro, M5 Pro/Max/Ultra)
- A single desktop GPU with 32GB+ VRAM (RTX 4090, RTX 5090, AMD Radeon)
- NVIDIA DGX Spark (native NVFP4 support)
Reported speeds from VentureBeat's coverage:
| Hardware | Decode Speed |
|---|---|
| M5 Ultra Mac Studio (96GB) | 210 tok/s |
| M5 Max MacBook Pro (48GB) | 125 tok/s |
| RTX 5090 | 104 tok/s |
| DGX Spark | 75 tok/s |
The fastest path to running it:
# Ollama (recommended for Mac)
ollama run inkling2mini
# llama.cpp / Llama App
llama serve -hf thinkingmachines/Inkling-2-Mini
# Docker
docker model run hf.co/thinkingmachines/Inkling-2-Mini
# vLLM (GPU servers)
vllm serve "thinkingmachines/Inkling-2-Mini"
Day-one support is unusually broad: Ollama, llama.cpp, Unsloth (GGUF quants), MLX, SGLang, vLLM, Exo Labs, NVIDIA Dynamo, TokenSpeed, and Docker Model Runner. The Hugging Face blog includes working code for Transformers, SGLang with speculative decoding, and MLX inference.
Quick start for Mac users: Install Ollama, then run
ollama run inkling2mini. That is the entire setup. The model downloads once (~20GB), runs entirely offline afterward, and needs no API key, no account, no cloud connection.
What the Community Is Saying
The community reception has been electric. Within hours of release, Inkling 2 Mini was trending number one on Hugging Face and number two was Unsloth's GGUF quantization of the same model.
Soumith Chintala, who tested it on his own DGX Spark and Mac Mini, wrote: "27B looping MoE, 3B active. Fits on a 32GB memory machine. Evals are roughly as good as Qwen 3.8 27B." His post drew 1,547 likes and kicked off a wave of independent testing.
Hugging Face CEO Clement Delangue posted: "American open-source AI isn't dead yet" — framing Inkling 2 Mini as proof that U.S.-built open-weight models can compete on the efficiency frontier that Chinese labs have dominated.
On r/LocalLLaMA, the 454-comment thread became a real-time benchmark party. Users reported successful runs on M3 Max MacBooks, RTX 4090 rigs, and even a used dual-RTX-3060 build. The 98.5% upvote ratio — unusually high for a model launch — suggests genuine satisfaction rather than hype.
The Hacker News thread (1,260 points, 253 comments) went deeper on the architecture, with particular interest in whether looping could be applied to existing models post-training.
Thinking Machines researcher @cHHillee noted that "more than 10% of one researcher (and his agents) for a couple weeks" drove most of the training — a striking claim about the efficiency of the development process itself. Hugging Face's Merve Noyan, who coordinated the day-one framework integrations, wrote: "Looping seems so intuitive I think it will open many horizons for other open models."
Why Looping Is the Real Innovation
Inkling 2 Mini is a good model. But the looped architecture is the story that matters beyond this single release.
The local-LLM community has spent the past year pushing against a wall: models below 30B parameters hit a quality ceiling that makes them unreliable for agentic work. The standard response has been Mixture-of-Experts — activate a small fraction of a much larger model. But MoE alone does not solve the core problem: each token still gets one pass through the network.
Looping changes that trade-off. By running middle layers multiple times, Inkling 2 Mini gets the reasoning depth of a much larger model while keeping the memory footprint of a small one. The official numbers claim that the looped version produces the same loss as a non-looped baseline at 60% of the inference compute. If those numbers hold up under independent testing, every other open-weight lab will be studying this architecture.
The companion variable-loop model is especially interesting for researchers. It lets you dial the loop count at inference time — more loops for harder problems, fewer for simple ones. Preliminary tests showed serial reasoning performance kept improving out to 10 loops, twice the training depth, with no increase in sabotage behavior. That is a novel property: a model that can trade speed for quality at inference time without retraining.
What This Means for You
If you are a developer running local models for coding assistance, Inkling 2 Mini is worth trying today. The setup is one command (ollama run inkling2mini), the model fits on mainstream hardware, and the Apache 2.0 license clears the way for commercial use. For agentic coding tasks — SWE-bench-style bug fixing, code generation, tool use — it matches or beats everything else in its size class.
If your work is research- or knowledge-heavy, wait. The factual recall gap is real, and the tendency to loop without converging on debugging tasks is a known weakness. Dense models like Qwen3.8-27B still win for multi-turn chat and complex constraint following.
If you are building products that run models on-device, pay attention to the architecture. Looping is a compelling primitive for edge deployment: it lets you ship a small model that punches above its weight class, then dial quality up or down based on the hardware available. Apple Silicon's unified memory makes looped models especially attractive — the same memory pool serves both the weights and the intermediate activations.
Three things to watch:
- Independent benchmarks. The numbers above are mostly from Thinking Machines' own evaluations. Wait for community-run evals on Terminal-Bench and Vals before committing to a production workflow.
- Looped fine-tuning. Can you fine-tune a looped model on domain-specific data without breaking the loop dynamics? Early support exists on Tinker, SkyRL, and Miles, but the community has not stress-tested this yet.
- The parent model. Inkling 2 Mini was distilled from an unreleased internal Inkling model. That parent is the real capability ceiling. When — and whether — it ships will determine if Thinking Machines is building a moat or sharing a floor.
The Bigger Picture
Futurum Group reports that 42% of enterprises now run open-weight models in production, up from 31% in July 2026. But 50% of pilots cite self-hosting costs as a hurdle. A model that delivers frontier-adjacent performance at 20GB on consumer hardware directly addresses that bottleneck.
For the open-weight ecosystem, Inkling 2 Mini proves that architectural innovation — not just scaling — can move the efficiency frontier. Looped transformers join MoE, speculative decoding, and quantization-aware training as tools that make local inference viable on hardware people already own.
For related coverage, see our guides on running AI locally in 2026, the M5 Ultra Mac Studio as a local inference rig, and our earlier coverage of Inkling and Kimi K3's open-weight frontier push. If the "local models are good enough" thesis interests you, check our analysis of Stanford's 71% finding.
The gap between cloud-only and runs-on-your-laptop just got smaller. And this time, the architecture — not just the benchmark score — is the reason.
Originally published at ComputeLeap
來源:DEV Community · dev.to



