如何記錄 LLM 請求日誌的元數據,而非訊息內容
Log the metadata, not the message
文章建議把 LLM 請求日誌分為元數據與內容,常規記錄元數據而不記錄提示詞和生成內容。元數據包括請求 ID、模型、token 用量、狀態、延遲、成本和時間;無法定價時,作者提醒不要在客戶端把缺失成本預設為零。作者亦提醒,複製日誌前先遮蓋內容,並將操作審計記錄與請求日誌分開保存。
Request logs are the first thing you reach for when something breaks and the last thing you want leaked. They are your best debugging tool and a concentrated store of exactly the data you told customers you would protect. The resolution is not a philosophy. It is a field list.
Split the log in two: metadata and content.
Metadata: log it generously
- Identity — request ID, key, organization. Which call was this, and whose?
- Shape — model, provider, prompt / completion / total tokens.
- Outcome — status, latency, cache-hit status.
- Money — the settled cost of that one call.
- Time — start and end timestamps.
Cost by model, latency percentiles, error regressions, cache effectiveness, per-key spend — all answerable from that list, without a single character of prompt text. The standards agree: OpenTelemetry's GenAI span convention requires the operation and provider names, recommends the model, finish reasons, response ID and usage token pair, and treats message content as opt-in because it is likely to contain sensitive information.
Content: a decision, not a default
The prompt and the completion buy you a narrow slice of extra value — reproducing one specific bug, sampling for quality review — and nearly all of the risk. On the gateway in the guide below, content is never written to the log; metadata always is, because metadata is the billing record. The data policy changes what the dashboard shows, not what is stored, and it is an admin-only lever. Full-content logging is a roadmap item, and the honest interim rule is: if you need the exact prompt weeks later, capture it on your own side, deliberately, with your own redaction.
Three disciplines that make the log trustworthy
- Capture the request ID. Every response carries it in a header. Store it beside your own trace ID on the success path and the exception path. Skip this and every investigation degrades to searching by timestamp.
-
Absent is not zero. When a call cannot be priced, the cost header is omitted and a status header says
unpriced. Default the missing value to zero in your client and your spend dashboard drifts from the ledger forever, silently. - Redact before you copy. CSV exports, forwarded headers, a second observability stack — your copy can carry content too. Redaction after the copy is theatre.
And keep the categories apart: who changed a budget or rotated a key belongs to the audit trail, a separate store with different volume and retention. Retention itself is a policy with an owner — bounded windows, exports for what must outlive them. Keep aggregates long, raw rows short, content shortest of all.
The full guide — field table, the four-step debugging workflow, the gateway-logs-versus-self-hosted-traces comparison, and a five-step verification routine — is here: https://nrouter.ai/blog/guides/llm-observability-request-logs
來源:DEV Community · dev.to