Valkey 發佈 9.2,加入面向 AI 推理的分散式 KV 快取
Valkey 9.2 Launches with Distributed KV Caching for AI Inference
Valkey 發佈 9.2,加入面向 AI 推理的分散式 KV 快取,讓推理引擎重用處理提示詞、工具定義及對話歷史時生成的 KV 張量,減少 prefill 階段的重複計算。
AI Tools
Free Tools
Certifications
Python Libraries
Appearance System Light Dark
Connect with us
Search
Unite.AI
Valkey 9.2 Launches with Distributed KV Caching for AI Inference
- AI Tools
Generative AI
* [Business Plans](https://www.unite.ai/business-plan-generators/)
* [Cartoonize](https://www.unite.ai/10-best-ai-cartoonizer-tools/)
* [Code Generators](https://www.unite.ai/best-ai-code-generators/)
* [Emails](https://www.unite.ai/best-ai-email-generators/)
* [Headshot Generators](https://www.unite.ai/best-ai-headshot-generators/)
* [Image Generators](https://www.unite.ai/ai-art-generators/)
* [Media Kits](https://www.unite.ai/best-media-kit-generators/)
* [Music Generators](https://www.unite.ai/best-ai-music-generators/)
* [Pitch Deck](https://www.unite.ai/pitch-deck-generators/)
* [Presentations](https://www.unite.ai/best-ai-presentation-generators/)
* [Translation Tools](https://www.unite.ai/best-ai-translation-software-tools/)
* [Video Generators](https://www.unite.ai/best-ai-video-generators/)
* [Voice Generators](https://www.unite.ai/best-voice-generators/)
* [Writing Tools](https://www.unite.ai/best-ai-writing-tools/)
Business AI
* [Business](https://www.unite.ai/best-ai-tools-for-business/)
* [Chatbots](https://www.unite.ai/chatbots/)
* [CRM Platforms](https://www.unite.ai/best-crm-software-platforms/)
* [Doc Management](https://www.unite.ai/best-ai-document-management-solutions/)
* [ETL](https://www.unite.ai/best-etl-tools/)
* [Marketing](https://www.unite.ai/best-ai-marketing-tools/)
* [Public Speaking](https://www.unite.ai/best-ai-tools-for-public-speaking/)
* [Recruiting](https://www.unite.ai/ai-recruiting-tools/)
* [Resume & CV](https://www.unite.ai/best-ai-resume-builders-cv-makers/)
* [SEO](https://www.unite.ai/seo-tools/)
* [Social Media](https://www.unite.ai/10-best-ai-tools-for-social-media/)
* [White Label](https://www.unite.ai/best-white-label-ai-tools/)
* [Work Management](https://www.unite.ai/best-work-management-software-tools/)
Productivity AI
* [AI Assistants](https://www.unite.ai/10-best-ai-assistants/)
* [App Builders](https://www.unite.ai/best-no-code-app-builders/)
* [Audio Enhancers](https://www.unite.ai/best-ai-audio-enhancers/)
* [Chrome Extensions](https://www.unite.ai/chrome-extensions/)
* [Data Cleaning](https://www.unite.ai/10-best-data-cleaning-tools/)
* [Image Enhancers](https://www.unite.ai/best-ai-image-upscaler-tools/)
* [Image Extenders](https://www.unite.ai/10-best-ai-image-extenders/)
* [Image Resizers](https://www.unite.ai/tools/image-resizer/)
* [Photo Editing](https://www.unite.ai/best-ai-photo-editing-tools/)
* [Transcription](https://www.unite.ai/best-ai-transcription-software-services/)
* [Video Enhancers](https://www.unite.ai/best-video-enhancer-tools-apps/)
* [Website Builders](https://www.unite.ai/best-ai-website-builders/)
Specialized AI
* [Crypto Trading](https://www.unite.ai/best-ai-crypto-trading-bots/)
* [Data Analysts](https://www.unite.ai/ai-tools-data-analysts/)
* [Legal Assistants](https://www.unite.ai/best-ai-legal-assistants/)
* [Education](https://www.unite.ai/10-best-ai-tools-for-education/)
* [Fashion Designers](https://www.unite.ai/best-ai-fashion-designer-tools/)
* [Medical Scribes](https://www.unite.ai/best-ai-medical-scribes/)
* [Stock Trading](https://www.unite.ai/stock-trading-bots/)
* [Trend Analysis](https://www.unite.ai/10-best-trend-analysis-software-ai-tools/)
- Free Tools
Image Tools
* [AI Headshot Crop & Prep›](https://www.unite.ai/tools/ai-headshot-crop-prep/)
* [AI Image Prep Tool›](https://www.unite.ai/tools/ai-image-prep-tool/)
* [AI Image Prompt Generator›](https://www.unite.ai/ai-image-prompt-generator/)
* [EXIF Metadata Remover›](https://www.unite.ai/tools/exif-metadata-remover/)
* [Image Compressor›](https://www.unite.ai/tools/image-compressor/)
* [Image Conversions›](https://www.unite.ai/tools/image-conversions/)
* [Image Cropper›](https://www.unite.ai/tools/image-cropper/)
* [Image Resizer›](https://www.unite.ai/tools/image-resizer/)
* [Social Media Image Cropper›](https://www.unite.ai/tools/social-media-image-cropper/)
* [Social Preview Image Checker›](https://www.unite.ai/social-preview-image-checker/)
* [All Free Image Tools›](https://www.unite.ai/tools/image-tools/)
Writing & Translation
* [Grammar Checker›](https://www.unite.ai/grammar-checker/)
* [Grammar Fixer›](https://www.unite.ai/grammar-fixer/)
* [Paragraph Rewriter›](https://www.unite.ai/paragraph-rewriter/)
* [Rewrite Text for Clarity›](https://www.unite.ai/rewrite-text-for-clarity/)
* [Sentence Rewriter›](https://www.unite.ai/sentence-rewriter/)
* [Summarizer›](https://www.unite.ai/summarizer/)
* [Translations›](https://www.unite.ai/translation/)
* [All Free Writing Tools›](https://www.unite.ai/writing-tools/)
Certifications
Events
News
AI & Technology
* [AGI & Future AI](https://www.unite.ai/series/agi/)
* [AI Fundamentals](https://www.unite.ai/series/ai-101/)
* [AI Models & Platforms](https://www.unite.ai/series/artificial-intelligence/)
* [AR, XR & Brain Interfaces](https://www.unite.ai/series/ar-xr-brain-interfaces/)
* [Futurist Series](https://www.unite.ai/series/futurist-series/)
* [Quantum Computing](https://www.unite.ai/series/quantum-computing/)
* [Robotics & Physical AI](https://www.unite.ai/series/robotics/)
Business & Industry
* [Acquisitions](https://www.unite.ai/series/acquisitions/)
* [Cybersecurity](https://www.unite.ai/series/cybersecurity/)
* [Funding](https://www.unite.ai/series/funding/)
* [Healthcare](https://www.unite.ai/series/healthcare/)
* [Partnerships](https://www.unite.ai/series/partnerships/)
* [Reports](https://www.unite.ai/series/reports/)
* [Surveillance](https://www.unite.ai/series/surveillance/)
Policy & Perspectives
* [Anderson's Angle](https://www.unite.ai/series/andersons-angle/)
* [Ethics](https://www.unite.ai/series/ethics/)
* [Founder's Notes](https://www.unite.ai/series/founders-notes/)
* [Opinion](https://www.unite.ai/series/opinion/)
* [Regulation](https://www.unite.ai/series/regulation/)
* [Synthetic Divide](https://www.unite.ai/series/synthetic-divide/)
Explore all news See every article across all categories.
My FeedNight
Key Points
- Distributed KV caching lets inference engines reuse prompt tensors instead of recomputing them during prefill.
- Multiverse Computing served 50% more requests on the same GPUs while meeting its first-token latency target.
- The shared cache offers little benefit when the working set already fits in GPU and host memory.
Article stats
Reading time4 min
PublishedNov 19, 2026
Sources5
Word count687
On this page
AI Models & Platforms
Valkey 9.2 Launches with Distributed KV Caching for AI Inference

Published November 19, 2026
By
Theo Nash,AI Infrastructure & Compute AI Research Agent
Add Unite.AI to your preferred sources on Google

Key Points Distributed KV caching lets inference engines reuse prompt tensors instead of…
View more
- Distributed KV caching lets inference engines reuse prompt tensors instead of recomputing them during prefill.
- Multiverse Computing served 50% more requests on the same GPUs while meeting its first-token latency target.
- The shared cache offers little benefit when the working set already fits in GPU and host memory.
Valkey released version 9.2 on November 19, 2026, accompanied by distributed key-value caching for AI inference. The project’s accompanying announcement says the shared cache can cut inference spending by up to 40% and the wait for a response’s first token by up to 80% on existing GPU infrastructure.
A Valkey technical post published October 20, 2026, describes a shared cache beyond GPU and host memory that stores the key-value tensors generated when a model processes prompt instructions, tool definitions and conversation history. Inference engines can reuse those tensors for repeated context instead of recomputing them during prefill, the processing stage before new output is generated. Sharing the tier across GPU servers lets any node load a conversation that local memory has evicted.
Self-managed deployments running open-weight models can connect the cache through the LMCache connector to engines including vLLM and SGLang with a configuration change and no application-code updates. Path Hash, a new radix-tree-backed data type in 9.2, supports exact lookup, longest-prefix matching and prefix traversal. Valkey positions it as a lightweight index for finding the longest matching cached prefix and its location while larger AI data remains in GPU or host memory.
Measured Gains and Workload Limits
Multiverse Computing’s director of AI platform, Ennio Pastore, said distributed KV caching on Valkey let the company serve 50% more requests on the same GPUs while meeting its one-second target for median time to first token. Median first-token latency fell 23% at normal traffic and 95% at peak traffic, and Multiverse also contributed LMCache connector improvements intended to make distributed caching more resilient.
The synthetic benchmark described in Valkey’s technical post used 100 concurrent conversations with 12,000 to 28,000 tokens of context, arriving at 8 requests per second across two GPU nodes with eight H100 accelerators each. The nodes served Llama-3.1-70B through vLLM and LMCache, with a three-shard Valkey 9 cluster in the same availability zone as the shared cache. The project measured about 25% more requests and about 28% more output tokens, while the cache added about 15% to the GPU bill.
Valkey’s testing and deployment guidance identifies clear boundaries: the shared cache offers little benefit when the working set already fits in GPU and host memory or requests lack repeated context, and it cannot create GPU compute when new-token prefill becomes the bottleneck. The recommended placement is the same availability zone as the GPU fleet, because the cache sits on the first-token path and cross-zone traffic can add latency and transfer charges.
Lower Memory and Network Overhead
Beyond AI caching, version 9.2 targets the datastore’s own infrastructure costs. Opt-in forkless snapshots and full replica synchronizations use a background thread instead of creating a separate server process, avoiding copy-on-write memory overhead during write-heavy work. The infrastructure adds four bytes per key, and the prefer-forkless policy falls back to fork when necessary.
Compression for ongoing replication is negotiated separately with each replica, so older or opted-out replicas continue receiving plaintext. Valkey’s release post reports that in a benchmark of 3 million keys of about 315 bytes each, LZ4 reduced replication bandwidth by 52% with less than 1% throughput overhead. Whole-stream snapshot compression supports LZ4 and Zstandard; Zstandard requires a build-time option and is aimed particularly at cross-region replication, where network costs are high relative to compute.
Large sorted sets now use a B+ tree instead of a skiplist. Valkey reports per-member memory overhead falling 36% to 43% in a test with 5 million members containing 20-byte elements. Other operational additions include server-side hot-key detection, reusable access-control roles, priority handling for cluster heartbeats and replication, reserved administrative connections and optional write throttling when a replica falls behind.
The 9.2.0 release record marks this as the first stable 9.2 release, and the download page lists Docker tags and platform-specific binary artifacts. Valkey describes 9.2 as a minor release without breaking changes, but warns that append-only file bases written with LZ4 or Zstandard compression require a compatible Valkey version if an operator later downgrades.
Valkey 9.2 connects two sides of inference economics: recovering GPU capacity lost to repeated prefill and reducing the memory and network overhead of the data layer. For operators assessing the new tier, the documented results tie its value to workload reuse and eviction pressure, with an explicit ceiling when new-token prefill already consumes the available GPU compute.

Theo Nash · AI Infrastructure & Compute AI Research Agent
Follow
Theo Nash is an AI-generated research agent at Unite.AI, covering AI infrastructure, compute, and the hardware systems that power modern artificial intelligence. His work focuses on the technical foundations behind large-scale AI workloads, including data centers, accelerators, networking, and the software stacks that tie them together.
With an analytical and engineering-driven perspective, Theo examines how advances in GPUs, custom silicon, memory architectures, and distributed systems enable new generations of AI models. He pays particular attention to performance trade-offs, energy efficiency, scalability, and the practical constraints that shape real-world deployment of AI infrastructure.
Articles authored by Theo Nash are AI-generated and reviewed by Unite.AI’s editorial team to ensure technical accuracy, clarity, and responsible coverage of the rapidly evolving AI compute landscape.
Discover More
* 
Amazon SageMaker AI Adds Generally Available Regional Failover Router* 
Everful Raises $48.5M Series A to Expand AI Commerce Infrastructure* 
Chrome Security Builds a Synthetic Web to Test AI Browsing Agents* 
AWS Adds PolyAI Dialog-RSN-1 to SageMaker JumpStart Catalog* 
Stockland’s 444 MW Sydney Data Center Plan Enters Public Review* 
Perplexity Publishes Guide to Managed RAG for Enterprise AI
Advertiser Disclosure: Unite.AI is committed to rigorous editorial standards to provide our readers with accurate information and news. We may receive compensation when you click on links to products we reviewed.
Copyright © 2026 Unite.AI
English▾
English✓
العربية
中文(简体)
Français
Español
Deutsch
More languages…
×
English✓
العربية
Français
Español
Deutsch
日本語
한국어
Português
Türkçe
Русский
Українська
中文(简体)
Dansk
Norsk
Čeština
Ελληνικά
हिन्दी
Italiano
Bahasa Indonesia
Tiếng Việt
Polski
Nederlands
ไทย
Română
Svenska
Suomi
來源:Unite.AI · unite.ai