跳到正文
DEV Community· devs·· 18 小時前AI 評分68

Thinking Machines Lab 發佈 Inkling 2 Mini 開放權重模型,WASAPI 雙音軌仍主導即時疊加延遲

Inkling 2 Mini Lands Open Weights on Laptop Hardware — Dual-Track WASAPI Still Sets the Overlay Floor

AI 導讀

Thinking Machines Lab 於 2026 年 11 月 20 日發佈 Inkling 2 Mini,並以 Apache 2.0 授權釋出開放權重。

正文

On 20 November 2026 Thinking Machines Lab shipped Inkling 2 Mini. It is a 26.4 billion parameter mixture-of-experts model with roughly 3.4 billion active parameters per token (5.4 billion effective once the middle layers loop three times). The checkpoint is about 20 GB in mixed NVFP4/MXFP8, runs on a single DGX Spark, an Apple Silicon machine with 32 GB+, or a decent discrete GPU, and is released under Apache 2.0. It takes text and images, has a 1 million token context, and posts strong numbers on agentic coding suites (DeepSWE v1.1, Vals Index, CyberBench) relative to other open models under 100 B total parameters.

Those numbers are real and useful. A model that small and capable is finally practical for on-device suggestion generation, short translation polish, or reply drafting without shipping every token to a remote endpoint. For a developer building a local overlay that shows translated suggestions during a technical interview or a sales call in another language, the model is one half of the stack.

The other half is still the Windows audio path.

Waveforms

The dual-track capture path still dominates

A usable live overlay needs two clean PCM streams:

  1. Local microphone (what you say).
  2. Loopback of the render device (what the other party says through Teams, Zoom, Meet, or the browser).

Both go through the Windows Audio Engine (audiodg.exe + audioeng.dll) in shared mode so the call client keeps control of the device. Exclusive mode would break the call. Typical shared-mode period is 10 ms. You can request a smaller period with IAudioClient3; the engine frequently ignores it or falls back under load. Communications enhancements, spatial audio, vendor APOs, and Bluetooth HFP/A2DP add another 10–40+ ms and become non-deterministic when the CPU or GPU is busy.

A practical stack looks like this:

  • Endpoint device → WASAPI shared-mode client (10–20 ms buffer).
  • Audio engine mixing and any enabled enhancements.
  • Capture callback (copy PCM, do minimal work, return immediately).
  • Framing + optional local VAD or silence detection.
  • Streaming STT (local GGUF or BYOK cloud).
  • Local model (Inkling 2 Mini or smaller) for translation polish or short suggestion.
  • Overlay paint on a dedicated thread (<16 ms if you stay off the UI thread).

The model inference cost for a short suggestion is now low enough that it no longer dominates. The first 20–50 ms are still spent inside the audio engine before any sample reaches a model. Add network RTT if the STT is remote, framing, and the time for the first usable partial, and the overlay still appears tens to hundreds of milliseconds after the speaker finishes. That is the floor.

Spectrogram

Why dual-track matters more than the model name

Single-stream capture (mic only, or loopback only) is useless for a bilingual call. You need both sides so the translator and the suggestion engine see the full conversation. Mixing them in one buffer loses speaker identity and makes barge-in detection harder. Separate tracks let you:

  • Run independent VAD or energy gates.
  • Apply different gain or noise profiles.
  • Feed context only for the remote side.
  • Keep the overlay responsive even when one track is quiet.

On Windows the cleanest way is two IAudioClient instances: one for the capture device, one for the loopback of the default render device (or the device the call client is using). Open the loopback client with AUDCLNT_STREAMFLAGS_LOOPBACK. Keep the callback non-blocking. Copy into a lock-free SPSC ring, convert format and resample off the audio thread, then hand the frames to STT.

A minimal sketch of the loopback open looks like this (error handling omitted):

IMMDevice* renderDevice = nullptr;
enumerator->GetDefaultAudioEndpoint(eRender, eConsole, &renderDevice);

IAudioClient* loopbackClient = nullptr;
renderDevice->Activate(__uuidof(IAudioClient), CLSCTX_ALL, nullptr, (void**)&loopbackClient);

WAVEFORMATEX* format = nullptr;
loopbackClient->GetMixFormat(&format);

REFERENCE_TIME hnsRequested = 100000; // 10 ms
loopbackClient->Initialize(
    AUDCLNT_SHAREMODE_SHARED,
    AUDCLNT_STREAMFLAGS_LOOPBACK | AUDCLNT_STREAMFLAGS_EVENTCALLBACK,
    hnsRequested, 0, format, nullptr);

loopbackClient->Start();

The same pattern, without the loopback flag, opens the mic. Two independent clients, two rings, one consumer thread that frames and dispatches.

Tradeoffs that actually ship

  • Shared mode + short period + separate tracks + non-blocking callback is still the architecture that works with a live call client.
  • Local models like Inkling 2 Mini move the suggestion/translation step on-device and remove one network hop. They do not rewrite the first half of the pipeline.
  • Bluetooth headsets, communications enhancements left on, or a busy audio engine still push the number higher. Measure the full path on your hardware; vendor numbers stop at their WebSocket or at the model’s time-to-first-token.
  • Process-wide audio ducking and communication-device policies inject extra silence or force resampling. Turn them off for the capture process if you can.

A typical end-to-end budget on a well-tuned machine:

Stage Typical cost
WASAPI shared period + callback 10–30 ms
Ring copy + format convert + resample + VAD 2–8 ms
STT first partial (local or remote) 40–120 ms
Local model suggestion (Inkling 2 Mini or smaller) 30–150 ms
Overlay paint <16 ms

Total from end-of-speech to visible text is usually 150–400 ms. The model release does not change the first term.

What actually moves the needle

  • Keep both tracks separate and time-aligned.
  • Stay in shared mode and request the smallest period the driver will honor.
  • Move everything after the callback off the audio thread.
  • Prefer a local or nearby STT endpoint when the network RTT is the next bottleneck.
  • Use a small on-device model only for the polish/suggestion step; do not block the audio path on it.
  • Measure with ETW or LatencyTest on the actual machine under load. DPC spikes still starve the audio engine.

Inkling 2 Mini is a real step for local agentic and suggestion workloads. It fits on hardware that already sits on a desk. It does not remove the Windows audio engine from the critical path of a live overlay that has to hear both sides of a call and paint text before the next sentence finishes.

I'm building Talkaside, a Windows app for job interviews and calls in another language.
Drafted with AI assistance from this project's code and logs.

來源:DEV Community · dev.to