跳到正文
Hacker News 熱門(buzzing.cc 中文翻譯)·· 12 天前AI 評分54

Flide 為 Jev-style 決策模型提供高速推理基礎設施

Show HN: Flide:让您的决策模型运行极快

AI 導讀

Flide 為 Jev-style 決策模型提供快速推理基礎設施,透過 exact state forking,速度最高可達 vLLM 和 SGLang 的 9.5 倍,且每個答案均與模型單獨回答該問題時一致。

正文

Flide: Extremely Fast Decision Inference with Exact State Forking

English · 简体中文 · 日本語

arXiv PyPI Website Docs License Python PyTorch

News · Quick start · Models · Options

Flide is fast inference infrastructure for Jev-style decision models. With exact state forking it is up to 9.5 times faster than vLLM and SGLang, and every answer is the one the model gives the question alone. It serves the Jev /v1/systemone API.

9.5 times faster than vLLM and SGLang; exact state forking; extremely fast

One state forked into three questions, each answered in one forward pass

🔥 News

Earlier news

Quick start

See the docs for every model, images, the API and all options.

uv venv -p 3.12 && source .venv/bin/activate
uv pip install flide torchvision
flide serve --model OmniJev/OneJev1.6-0.8B

Any model in the model table starts the same way, for example Perplexity's decision model.

flide serve --model perplexity-ai/pplx-decider-flash-v1.2

Every one of them answers this request.

curl http://localhost:8000/v1/systemone -H "Content-Type: application/json" -d '{
  "state": "Order #1182 arrived with a cracked screen. The customer asks for a refund and attached two photos.",
  "questions": {
    "action": {"type": "choice", "instructions": "What should support do next?",
               "criteria": {"refund": "refund the order", "replace": "send a replacement", "escalate": "ask a human"}},
    "urgent": {"type": "noul", "instructions": "Does this need a reply within the hour?"}
  }
}'

Models

The last three columns say how much faster Flide answers one request than the engine its release ships, as the median latency over the requests of each workload on one H200, where DI is the Decision Index. The chat models run against the faster of vLLM and SGLang, and OneJev against qev, the engine it used before Flide. KnowLine 35B runs with --quantize fp8, as its release serves it in FP8. A value marked ~ is the average over the sizes tested.

Model Inputs DI MultiRC QuALITY
Ainvest-Decision text 1.4× 1.3× 1.3×
AJev text 3.7× 9.3× 2.6×
AutoTrust text, images ~3.2× ~7.6× ~1.6×
Bespoke-Nimble text 2.6× 2.1× 1.6×
cerno text ~4.1× ~8.6× 1.1×
Clef text, images 1.3× 1.0× 1.0×
Darwin text ~1.4× ~4.2× ~1.2×
Drex text, images ~2.8× ~6.2× ~1.5×
Gemma 4 text, images ~1.3× ~1.7× ~1.0×
Intern-Decision text, images 4.4× 1.1× 1.5×
JADE text 2.0× 9.2× 1.7×
Jeffan text 3.1× 2.6× 1.9×
JevAny text, images 1.5× 5.8× 1.2×
jiwo text ~1.9× ~5.9× 1.8×
Kev text ~7.9× ~1.5× ~1.7×
KnowLine text, images ~1.4× ~4.3× ~1.5×
Linne text 2.3× 8.6× 1.2×
Lumma text, images 1.7× 7.1× 1.2×
Lychee text 1.4× 1.5× 1.4×
mammoth text, images ~4.3× ~2.8× ~1.8×
mogan text 1.6× 8.5× 1.8×
OneJev text, images, video ~6.8× 7.3× 4.7×
Open-Jev text 4.2× 5.0× ~1.5×
pplx-decider text, images ~2.2× ~1.4× ~1.3×
Quyet text ~3.9× ~7.1× ~1.6×
Qwen3.5 text, images ~2.1× ~4.8× ~1.3×
Senova text 3.1× 28.1× 1.2×
Sieve text 3.0× 14.3× 2.6×
StartLux text 2.6× 1.1× 1.2×
SynACK Decide text 3.3× 11.9× 3.0×
torchcast text, images 1.6× 8.4× 1.4×
Wald text 1.6× 12.7× 1.5×
Xor text, images, video 2.5× 6.2× 2.8×

Options

Option What it does
--session-cache-gb 40 keeps the states of earlier requests, so an agent's next step reads only what is new
--batch-invariant the same bits for a request whatever it is batched with, at some cost in speed
--quantize fp8 Qwen3.5 in FP8 on Ada, Hopper and Blackwell GPUs
--frontends 2 HTTP, rendering and tokenizing in two processes of their own, for many clients

License

Apache-2.0. flide/kernels/gdn.py holds a kernel derived from two kernels of flash-linear-attention (MIT); THIRD_PARTY_NOTICES lists it with the other code Flide builds on.

Citation

If you use Flide in your research, please cite our paper.

@article{li2026flide,
  title   = {Flide: Accelerating Decision Inference with Exact State Forking},
  author  = {Li, Bobo and Song, Jiajia and Fei, Hao and Hsu, Wynne and Lee, Mong Li},
  journal = {arXiv preprint arXiv:2610.42148},
  year    = {2026}
}

來源:Hacker News 熱門(buzzing.cc 中文翻譯) · github.com