模型在 type 前輸出 content 時的 JSON 串流處理

Streaming JSON when the model emits content before type

AI 導讀

Claude 在 type 前輸出 content 時，解析器會暫存串流尾端，待識別欄位後再輸出內容。解析器每次輸出 20 characters，或每隔 100 ms 輸出；若串流開始時仍未出現 type，會在 finalize 時補上，title 則會在 content 結束後附加。

正文

Suggestions stream as JSON with three fields: type, title, and content. OpenAI and Google calls send response_format set to json_schema. The OpenAI-compatible Anthropic layer has no response_format. JSON is an instruction in the system prompt, and on that path Claude can emit content before type.

A parser that prints the first string value will show the rest of the object as the suggestion. The braces at the end are still syntax.

The parser is a state machine with a short reserve. It holds the tail until the key is identified, then flushes 20 characters or every 100 ms. If type is still missing when the stream starts, it is filled in at finalize. title can arrive after content has already ended, so it is attached then.

The same parser runs in LiveSuggest before a suggestion is shown.