Strot 把 LLM 決策任務編譯成可本機執行的獨立管線

Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Show HN: Strot – Compile an LLM's intent once, run decisions locally (github.com/strot-ai) 22 points by solohunter0011 hour ago | hide | past | favorite | 10 comments help publlus_enigma10 minutes ago | next[–] I love the concept, but tried reading the README on the got, but I am still unclear about precisely what it does or how it does it. Some more detail would be welcome. reply wonnage24 minutes ago | prev | next[–] > If a ghost is close (nearest_ghost_distance small, " "within 2 steps), choose flee_ghost. Otherwise choose " "chase_dot to eat the nearest dot." Dude, this is AI psychosis. Just ask the LLM to write you a program (“compile”) and then you can run it locally forever reply ghurtado5 minutes ago | parent | next[–] That is literally what this does reply jMyles44 minutes ago | prev | next[–] I think that "infer once, repeat resulting logic forever" is one of the most important / sustainable / humane ways to imagine how AI acclimates to the universe. https://pickipedia.xyz/wiki/PickiPedia:Artificial_Dumb reply solohunter0018 minutes ago | parent | next[–] I was working with a large entries on Batch inference, and yes LLMs are quite expensive to process Billions of contracts at latency, so I in my orignial paper was on the expensive thinking once, run the cheap deterministic result forever. reply solohunter0011 hour ago | prev[–] Hi HN. STROT compiles a decision task — "route these tickets", "score this feedback", "classify this text" — into a standalone pipeline. A large LLM is used once, at compile time, to synthesize the labels, the input projection, and the handlers. At runtime a small local model (144M, CPU) makes every decision: ~60ms/record, $0 marginal, zero LLM calls, bounded typed-JSON output. The goal was to move the expensive, nondeterministic part (an LLM) entirely to build time and keep runtime frozen, cheap, local, and auditable — the way a compiler moves work off the hot path. What I think is actually interesting, beyond "small model does inference": - The compiler measures instead of guessing. It scores candidate labelings on a teacher-labeled sample and keeps the winner; on the identical model that's +18pp on a routing benchmark (92.5% vs 74.8%) and +14pp on text classification (86.7% vs 72.7%) over naive integration. - It fits a conformal-calibration threshold at compile time, so low-confidence records abstain with a marginal error guarantee rather than guessing. - A golden-record gate rejects any compile that fails known-answer examples — synthesis is stochastic, and I show that variance in the benchmarks. What it doesn't do: it can't make a small model understand things it fundamentally can't. On tweet hate-speech it hits the model's ceiling; the fix there is a task-trained model in the swappable slot, not the compiler. All of it — including the losses — is in the repo's benchmarks with reproducible scripts. There's a browser demo (a 144M model ( which is changeable ) playing Pac-Man, one decision per frame) and a quickstart that runs in a few lines. MIT licensed, pure-stdlib core. Happy to go deep on the calibration math, the compile/cache design, or where this does and doesn't make sense. reply razingeden11 minutes ago | parent | next[–] >On tweet hate-speech it hits the model's ceiling This sounds like a sus endeavor to redline your LLM with, could you elaborate ? reply sysguest43 minutes ago | parent | prev | next[–] > Happy to go deep on the calibration math, the compile/cache design, or where this does and doesn't make sense. yes plz reply dpacmittal32 minutes ago | parent | prev[–] Was this entirely vibe coded? The readme is very obviously Claude with no human edits at all. reply nodja15 minutes ago | root | parent[–] Yes, even the comment you replied to is very noticeably claude authored. reply Consider applying for YC's Winter 2027 batch! Applications are open till November 2. Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact Search: