Harish Kotra 以 Perplexity 的 pplx-decider-flash-v1.2 搭建會標示不確定性的決策工具

A 4B decision model on a Mac Mini, and the 30 lines of UI that make it honest.

Ask a chatbot "should I take this job?" and you get three paragraphs. They will be well written, they will mention trade-offs, and they will quietly commit to an answer that the model did not actually compute. The prose manufactures confidence. That is the failure mode nobody puts in the eval table.

So when Perplexity released pplx-decider-flash-v1.2, a decision model with the language head removed, the interesting thing was not the benchmark score. It was the shape. This model cannot write a paragraph. It takes a state, answers typed questions about it, and returns calibrated probabilities. Zero output tokens. One prefill pass per read.

That is a machine for honest answers. I built a public oracle around it and spent most of the time on the opposite of a normal ML project: not making the number better, but making the interface refuse to oversell it.

What the model actually is

The checkpoint is 4B parameters, distilled from Perplexity's 27B decision models. The backbone is Qwen3.5-4B. The language head is gone, replaced by a linear readout over 255 answer codes, each code mapping to a token id.

You do not send a prompt. You send a request:

{
  "state": "My integration keeps failing with a 401 since we rotated our API key.",
  "questions": {
    "urgent": { "type": "noul", "instructions": "Is the customer blocked right now?" },
    "team": {
      "type": "choice",
      "criteria": { "billing": "Charges and refunds", "support": "Integration errors" }
    },
    "frustration": { "type": "score", "criteria": ["calm", "mildly annoyed", "frustrated", "angry"] }
  }
}

and you get probabilities back for every question at once, in a single forward pass, at the cost of reading the prompt.

The card also notes something that shaped the whole build: for this smaller checkpoint, one question per request is more accurate than grouping on most multi-question tasks, by up to 20 points on some categories. The 27B model behaves the other way. So the app asks four separate questions and batches them into one call.

Getting it running on an M4

The card says CUDA. I have an M4 Mac Mini with 64 GB of unified memory and no NVIDIA silicon anywhere in the house. Two things had to be true: the hybrid linear-attention backbone had to run on MPS, and the 8.5 GiB of BF16 weights had to fit without becoming 18 GB.

The first part was free. torch 2.14 with MPS loaded the Qwen3.5 class and produced logits on the first try. The second part needed a two line seam, because the vendor loader picks the dtype by device:

dtype = torch.bfloat16 if self.device.type == "cuda" else torch.float32

On a Mac that means float32, which doubles the footprint for no accuracy gain, since the weights ship in BF16. I did not patch their source. I wrapped the classmethod around the load and put it back afterwards:

def patch_qwen3_5_dtype(torch, target):
    from transformers.models.qwen3_5 import modeling_qwen3_5 as m
    original = m.Qwen3_5Model.from_pretrained.__func__

    def wrapper(cls, *args, **kwargs):
        kwargs["dtype"] = target
        return original(cls, *args, **kwargs)

    m.Qwen3_5Model.from_pretrained = classmethod(wrapper)
    return lambda: setattr(m.Qwen3_5Model, "from_pretrained", classmethod(original))

Load to GPU takes 5.2 seconds warm. A full read, four passes over roughly 750 input tokens, takes 2.7 to 3.7 seconds. The backbone falls back to reference kernels because flash-linear-attention and causal_conv1d are not available on macOS, and that is the honest reason a read is three seconds instead of three hundred milliseconds.

The first real surprise

The plan was simple: question in, probability out, bar underneath. I wrote the wrapper, asked it a few things, and immediately hit the thing that made the project worth building.

With no context, the model is confidently wrong in a very specific way.

Nothing in the text decides any of these. The model is returning a prior, and the prior leans yes, because the proposition being tested is phrased as "the answer is yes". A naive UI would render 93% in 130px type and call it a forecast. That is the exact failure the model's design was supposed to prevent, reintroduced by the front end.

The same model with real evidence behaves completely differently:

Same weights, same temperature. The number only means something when the text carries something to read.

So the card became a state machine

The fix was to stop treating the probability as the product and make the state the product. Four states, checked in this order:

function verdict(p, spread, evidence) {
  const d = Math.abs(p - 0.5);
  if (evidence < 0.15) return { label: 'guessing', kind: 'guess' };
  if (spread >= 0.18) return { label: 'unstable', kind: 'unstable' };
  if (d <= 0.06) return { label: 'coin flip', kind: 'flip' };
  const side = p > 0.5 ? 'yes' : 'no';
  if (d <= 0.12) return { label: `barely ${side}`, kind: 'lean' };
  if (d <= 0.25) return { label: `leans ${side}`, kind: 'lean' };
  return { label: side, kind: 'clear' };
}

guessing comes first on purpose. If the evidence score is under 0.15, nothing downstream matters: the bar turns grey, the accent colour is removed, and the card says the number is a prior, not a prediction. The most useful screen in the entire app is the one where it refuses.

Stability is the second honest signal

Calibration is a property of the checkpoint that I cannot verify in an afternoon. What I can measure, on every single read, is whether the answer survives being asked differently. So each question is asked three ways:

PHRASINGS = (
    'The answer to the question "{q}" is yes.',
    'If someone asks "{q}", the correct answer is yes.',
    'Taking the situation into account, "{q}" should be answered yes.',
)

p is the mean of the three, spread is max minus min. On evidence-backed questions the spread sits around 0.03 to 0.06. On evidence-free ones it regularly hits 0.16 to 0.24, and the card says so in plain language: "the three readings landed 24 points apart, so treat this as unstable."

This costs two extra passes and about two seconds. It is the cheapest credibility the app has.

The fourth pass is the evidence score, an ordinal question with four levels:

EVIDENCE_LEVELS = (
    "no evidence, only the question itself",
    "a vague hint, nothing checkable",
    "some relevant specifics",
    "concrete numbers, constraints and details",
)

Normalised to 0..1 and shown as none / a hint / some / strong. It is not a measure of truth. It is a measure of how much the model had to work with, which is the thing users actually need to know before they act on a number.

The screenshot is part of the design

An oracle that argues with you is only interesting if other people can see the argument. The share card is generated client-side on a 1080x1350 canvas: the question, the verdict word, the number, the bar with a notch at 50, one sentence of explanation, and the byline. No server round trip, no image library, no dependencies.

That choice matters more than it looks. The card carries guessing and unstable as loudly as it carries yes, so the thing people post when they are annoyed is also the thing that explains the model's limits.

What it costs

No API key, no per-token bill, no rate limit to design around. One process-wide lock serialises reads, which is the right amount of concurrency for a demo and an obvious thing to replace with a queue if this ever met traffic.

What I would build next

The missing piece is a calibration harness. Two hundred questions with known outcomes, predicted p on one axis and observed frequency on the other, published as a reliability curve. Until that exists, the calibration claim belongs to Perplexity's development set, not to my question distribution, and the README says exactly that.

After that: vision reads, since the checkpoint already accepts images up to 1.6 megapixels and "should I buy this listing?" with a screenshot is a two hour change. Then a decision journal, so the app can show you the reads where the model beat you and the ones where you beat it.

The whole project is five files, one checkpoint, and a SQLite table. It runs on a machine that sits on my desk and pulls about 40 watts.

If you build on it, keep the rule: no number without its evidence, and never let a state pretend to be a prediction.

Code & more: https://www.dailybuild.xyz/project/295-coin