JSON Schema 欄位順序如何令 vLLM 分類器先定標籤、後生成推理

My ticket classifier had a reasoning field, and the reasoning was good. It would write "the customer was charged twice for the same invoice, this is a billing problem" and then file the ticket under bug.

Not once. Dozens of times.

The Pydantic model was correct. reasoning came first, category second. I checked it three times. The bug was a cache helper that sorted my JSON schema keys alphabetically, and in the alphabet, category comes before reasoning. That one sort flipped the JSON schema field order, and the model started committing to a label before it had written a single word of thought.

TL;DR

• With constrained decoding (vLLM guided_json / response_format: json_schema, Outlines, xgrammar, llama.cpp grammars), JSON schema field order is generation order. The model writes the keys in the order the schema lists them.
• LLMs generate left to right. If category comes before reasoning, the label is fixed before any reasoning tokens exist. The "reasoning" becomes an after-the-fact excuse.
• On my 600 hand-labeled support tickets, label-first scored 491/600 (81.8%). Reasoning-first scored 541/600 (90.2%). Same model, same prompt, same temperature.
• Watch for anything that reorders your schema: json.dumps(sort_keys=True), YAML round-trips, schema "normalizers", hand-written dicts.
• Cheap detector: count outputs where the reasoning names a different category than the label. A high number means your fields are in the wrong order.

Does JSON schema field order change LLM output?

Yes. With grammar-constrained decoding, the backend builds a state machine from your schema and only lets the model emit tokens that keep the JSON valid. The backends I've used walk properties in the order they appear. So the first key in your schema is the first thing the model writes, every time.

That sounds like a formatting detail. It isn't, because of how generation works.

Why does field order matter for reasoning?

An autoregressive model produces one token at a time, and each token can only see the tokens before it. Nothing written later can reach back and change an earlier token.

So compare these two outputs:

{"category": "bug", "reasoning": "The customer was charged twice..."}

{"reasoning": "The customer was charged twice...", "category": "billing"}

In the first one, "bug" was chosen around token 5, with nothing in front of it except the prompt and {"category": ". All the analysis comes afterward and has zero influence on the label. The model is now writing a justification for a decision it already made, and sometimes the justification honestly disagrees with the decision.

In the second one, the label is generated with 40 to 80 tokens of the model's own analysis sitting right in its context. That's the whole point of asking for reasoning in the first place.

How did my schema get reordered?

Here's the model. It's fine:

from typing import Literal
from pydantic import BaseModel

class Triage(BaseModel):
    reasoning: str
    category: Literal["billing", "bug", "account", "feature_request", "other"]

Pydantic v2 keeps declaration order in model_json_schema(), so properties comes out as reasoning, then category. Good.

Then I wrote this, because I wanted to memoize compiled grammars and lru_cache needs hashable arguments:

import json
from functools import lru_cache

def build_request(schema: dict, prompt: str) -> dict:
    return _build(json.dumps(schema, sort_keys=True), prompt)

@lru_cache(maxsize=64)
def _build(schema_json: str, prompt: str) -> dict:
    schema = json.loads(schema_json)  # keys are now alphabetical
    return {
        "messages": [{"role": "user", "content": prompt}],
        "response_format": {
            "type": "json_schema",
            "json_schema": {"name": "triage", "schema": schema},
        },
    }

sort_keys=True is the reflex every Python developer has when they want a stable cache key. It sorts nested dicts too, which includes properties. json.loads preserves the order it reads, so the sorted order sticks.

category < reasoning. The model now answered first.

The fun part: the outputs still looked great. Valid JSON, a sensible label most of the time, a thoughtful paragraph underneath. Nothing errored. Accuracy just quietly dropped.

How much accuracy did the wrong field order cost?

On my set it cost 50 tickets out of 600, or 8.4 points. This is one dataset, one 14B-class instruct model on vLLM, temperature 0, same prompt for both runs. Your gap will be different. The direction is what I'd bet on.

The more telling number was in the failures. Of the 109 wrong answers in the label-first run, 47 had reasoning that explicitly named the correct category. The model knew. It just knew too late.

The tickets that flipped were the ambiguous ones: double charges caused by a checkout bug, login failures caused by an expired card, feature requests phrased as complaints. Easy tickets came out right in both orders. Thinking only helps when there's something to think about.

Why is label-first worse than no reasoning at all?

It can be, because of how the grammar masks tokens. At the label position, the decoder only allows tokens that can start one of your enum values. Every other token gets its probability set to zero, and what's left is renormalized.

An illustrative example (made-up numbers, real mechanism): say the model's next-token distribution right after "category": " puts most of its mass on tokens that would start a hedge or a compound label it can't actually emit. The grammar deletes all of those. Among the survivors, bug might have 0.06 and billing 0.04, so after renormalization bug wins with 60%. The model never "chose" bug in any meaningful sense. It was the least bad legal option at a moment when it hadn't worked anything out yet.

Then the reasoning field arrives with no constraint, the model writes what it actually thinks, and you get my double-charge ticket filed as a bug.

Prefix sharing makes it worse. If you have account and account_deletion, the first token decides which family you're in before the model has committed to anything specific.

How do you check whether your structured outputs answer first?

Three checks, all cheap.

1. Print the schema you actually send. Not the Pydantic model. The final dict, right before the HTTP call.

req = build_request(Triage.model_json_schema(), prompt)
print(list(req["response_format"]["json_schema"]["schema"]["properties"]))
# ['category', 'reasoning']  <- there it is

2. Look at the raw JSON text. Constrained decoders write keys in generation order, so the raw response string shows you what came first. If you only ever look at the parsed object, you'll never notice.

3. Count contradictions. A crude keyword check catches a lot:

CATS = ["billing", "bug", "account", "feature_request", "other"]

def contradicts(out: dict) -> bool:
    text = out["reasoning"].lower()
    named = {c for c in CATS if c.replace("_", " ") in text}
    return bool(named) and out["category"] not in named

rate = sum(contradicts(o) for o in outputs) / len(outputs)

In my label-first run this flagged about 8% of all outputs. After the fix it dropped under 2%, and most of those were the reasoning mentioning two categories while weighing them, which is fine.

What should the field order be?

Put anything the model needs to figure out before anything that depends on it. In practice:

1. reasoning (or evidence, analysis) first. Keep it short. Ask for two or three sentences in the prompt. Don't rely on maxLength being enforced, since support varies by backend.
2. The decision second. Label, score, yes/no.
3. confidence after the decision, if you keep it at all. A confidence emitted before the label is a number about nothing.
4. Extracted values in source order. If you pull fields out of a document, order them the way they appear in the text so the model reads forward instead of jumping around.

And stop reordering schemas by accident. If you need a cache key, hash a sorted copy and send the original:

import hashlib

def schema_key(schema: dict) -> str:
    return hashlib.sha256(json.dumps(schema, sort_keys=True).encode()).hexdigest()

_cache: dict[str, dict] = {}

def build_request(schema: dict, prompt: str) -> dict:
    _cache.setdefault(schema_key(schema), schema)  # keep the original order
    ...

Two costs to know about. Reasoning-first means more output tokens before you get your label, so latency goes up and streaming the label early is off the table. And if a response hits max_tokens, you lose the label, not the reasoning, so check the stop reason before you parse.

If you need the label fast and still want accuracy, run two passes: free-text analysis with no grammar, then a tiny constrained call that only extracts the label from that analysis.

So does field order matter in structured output?

Yes. With constrained decoding in vLLM, Outlines, xgrammar or llama.cpp grammars, JSON schema field order is the order the model generates the fields, and a model can't revise a token it already emitted. Put reasoning before category and the label is conditioned on the analysis. Put category first and the label is locked in before any analysis exists, so the reasoning turns into a justification that sometimes argues against its own answer. Declare reasoning first, print the final schema you actually send to make sure nothing like sort_keys=True reordered it, and count reasoning-versus-label contradictions to catch it in production.

Written by the developer behind Preterview, an interview prep platform.