跳到正文
DEV Community· TECHDEV·· 3 小時前AI 評分60

TECHDEV 開發 jsonfold,以可逆鍵名摺疊縮減提示詞中的 JSON

I was pasting 4KB of JSON into every prompt. So I built a key-folder that shrinks it — reversibly.

AI 導讀

TECHDEV 開發 jsonfold,將 JSON 中重複的長鍵名替換為短別名,並可透過映射還原原始內容。在作者自製的 3,760-byte API 回應示例中,資料縮至 2,940 bytes,節省 820 bytes(21.8%),約節省 205 個估算 tokens。工具不改動值,會執行往返檢查並提供映射收據;專案採 MIT 授權、零依賴,支援 Node 18+。

正文

I was pasting 4KB of JSON into every prompt. So I built a key-folder that shrinks it — reversibly.

Every time I debug against an API, the loop is the same: curl something, get a wall of JSON, paste it into a prompt, and watch a third of my context window evaporate into "repository_owner_login" repeated forty times.

jq -c helps — it strips the whitespace. But whitespace was never the real problem. The real problem is the keys. In a typical API response, object keys are 20–40% of the bytes, and the same long names repeat on every single object in every array.

There are lossy "AI compressors" that will happily rewrite your JSON into something smaller. I don't trust them: if the tool can't show me exactly what changed — and put it back, byte for byte — I'm not pasting its output anywhere near production data.

So I built the boring version: jsonfold. It folds long JSON keys into short aliases before the prompt, and unfolds them back afterwards. Lossless, reversible, and every run prints a receipt you can verify.

What it does

Three commands, zero dependencies, plain Node:

# fold: long keys -> $0, $1, ...  (receipt goes to stderr, JSON to stdout)
cat api.json | node bin/jsonfold.js fold > folded.json
# 3,760 -> 2,940 bytes, saved 820 bytes (21.8%), ~205 est. tokens
# 27 of 29 unique keys aliased (133 occurrences), round-trip OK

# unfold: restore the original, byte-identical
node bin/jsonfold.js unfold folded.json --map folded.json.map -o restored.json
diff api.json restored.json && echo IDENTICAL

# stats: see where the bytes actually go before you fold
node bin/jsonfold.js stats api.json

The aliasing is deliberate, not clever:

  • Aliases are assigned by bytes saved (frequency × length), so the most repeated long keys get the shortest aliases first.
  • A key is only aliased when the alias is strictly shorter than the key. Two-letter keys like "id" are left alone — aliasing them would make things bigger.
  • If your data already contains keys shaped like $0, the tool switches prefix (#, @, ~…) instead of colliding, and records the choice in the map file.
  • Every fold runs an internal round-trip check — fold, unfold, deep-compare against the original — and fails loudly if anything mismatches. The receipt isn't marketing; it's the tool checking its own work.

Token estimates use the rough ~4 chars/token heuristic and are labeled as such. I don't know your model's tokenizer, and the tool doesn't pretend to.

A real example

I made a 3,760-byte fixture shaped like a repo-list API response (invented data, 29 unique keys). Folding it:

3,760 -> 2,940 bytes, saved 820 bytes (21.8%), ~205 est. tokens
27 of 29 unique keys aliased (133 occurrences), round-trip OK

21.8% from key folding alone, on top of whatever jq -c already saved you. The folded JSON is still valid, readable-ish JSON — you can eyeball it, unlike dictionary-packed blobs.

What it is not

Honest comparison, because "yet another JSON tool" deserves skepticism:

Tool What it does What jsonfold does differently
jq -c Strips whitespace Also shortens keys — the bigger win on real payloads
jsonpack (npm) Dictionary-encodes keys and values into a flat ASCII string Keeps output as readable JSON you can paste into a prompt, plus a verifiable receipt
Lossy LLM compressors Rewrite/summarize your data smaller Never touches values; unfold restores the original byte-for-byte

It won't drop fields, reorder anything, or summarize. That's jq's job, or a conversation with the model. This tool does one thing: make the keys cheaper.

Try it

git clone https://github.com/techdevfun/jsonfold
cd jsonfold
node bin/jsonfold.js stats your-file.json

MIT licensed, no dependencies, works on Node 18+. The repo has 42 tests, CI on three Node versions, and a demo script whose output is diffed in CI.

Roadmap, if people find it useful: an npx release, a --inline mode that embeds the map for single-file handoffs, and per-key savings in stats. Issues and "this broke on my payload" reports welcome — especially weird-key edge cases.


Repo: github.com/techdevfun/jsonfold — MIT, zero dependencies.

來源:DEV Community · dev.to