C.C. 打造 sjq Rust CLI，逐事件解析 OpenAI 相容 AI 串流
I kept hitting jq: parse error on AI streams, so I built sjq
C.C. 打造 Rust CLI sjq，用於解析 OpenAI 相容的伺服器傳送事件（SSE）串流，並逐事件輸出精簡內容。它會顯示串流新增的內容，以及 TTFT、總耗時和 tok/s；透過 sjq curl 亦可列出連線、TLS 握手及 TTFB 時間。
If you work with LLMs, you've done this: curl -N <streaming endpoint> | jq. And you've seen this:
jq: parse error: Invalid numeric literal at line 1, column 9
Column 9. That's where data: ends and the JSON begins. An SSE stream isn't JSON — it's a framing protocol: data: {...} lines separated by blank lines, [DONE] at the end. jq never stood a chance.
So I built sjq — jq, but for AI streaming. A small Rust CLI that parses OpenAI-compatible server-sent event streams (/v1/chat/completions and /v1/responses) and prints one short line per event, showing only what each event added.
What it looks like
sjq curl -s $BASE/v1/chat/completions \
-H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
-d '{"model":"gemini-4-argon","stream":true,"messages":[{"role":"user","content":"hi"}]}'
model: gemini-4-argon
role: assistant
data: 'Hi'
data: ' there'
data: ' 👋'
…
time: ttft=0.42s total=3.1s tok/s=258.2
Instead of 99% envelope JSON, you watch the actual content arrive — plus a timing line: TTFT, total, tokens/sec. Run through sjq curl and it also breaks out connect, TLS handshake, and TTFB, which is genuinely useful when you're debugging a slow provider.
It understands tool calls (name + arguments as they stream in), reasoning_content, refusals, logprobs, usage blocks, and multiple choices — each rendered as its own labeled line.
Install
brew tap vivgrid/tap
brew trust vivgrid/tap # Homebrew 6+ requires trusting third-party taps
brew install sjq
Or from source: cargo install --git https://github.com/vivgrid/sjq (one dependency: serde_json).
Why not just strip the framing with sed?
You can — sed 's/^data: //' | jq — and plenty of people do. But then you're staring at full envelope objects per chunk: ids, timestamps, object: "chat.completion.chunk", finish_reason: null fifty times over. The signal (the tokens) drowns in the envelope. sjq inverts that: the delta is the output; the envelope gets summarized once.
MIT licensed, single binary. The repo ships 14 test fixtures covering the gnarly edges: multi-line data payloads, mid-stream errors, parallel tool calls, logprobs, CJK/emoji text.
Repo: https://github.com/vivgrid/sjq — feedback and weird streams welcome.
來源：DEV Community · dev.to