跳到正文
Unite.AI·· 1 小時前AI 評分63

Valkey 發佈 9.2,加入面向 AI 推理的分散式 KV 快取

Valkey 9.2 Launches with Distributed KV Caching for AI Inference

AI 導讀

Valkey 發佈 9.2,加入面向 AI 推理的分散式 KV 快取,讓推理引擎重用處理提示詞、工具定義及對話歷史時生成的 KV 張量,減少 prefill 階段的重複計算。

正文

Image 1: Unite.AI

Connect with us

Search

Image 2: Unite.AI

Unite.AI

Valkey 9.2 Launches with Distributed KV Caching for AI Inference

  • AI Tools

Generative AI

    *   [Business Plans](https://www.unite.ai/business-plan-generators/)
    *   [Cartoonize](https://www.unite.ai/10-best-ai-cartoonizer-tools/)
    *   [Code Generators](https://www.unite.ai/best-ai-code-generators/)
    *   [Emails](https://www.unite.ai/best-ai-email-generators/)
    *   [Headshot Generators](https://www.unite.ai/best-ai-headshot-generators/)
    *   [Image Generators](https://www.unite.ai/ai-art-generators/)
    *   [Media Kits](https://www.unite.ai/best-media-kit-generators/)
    *   [Music Generators](https://www.unite.ai/best-ai-music-generators/)
    *   [Pitch Deck](https://www.unite.ai/pitch-deck-generators/)
    *   [Presentations](https://www.unite.ai/best-ai-presentation-generators/)
    *   [Translation Tools](https://www.unite.ai/best-ai-translation-software-tools/)
    *   [Video Generators](https://www.unite.ai/best-ai-video-generators/)
    *   [Voice Generators](https://www.unite.ai/best-voice-generators/)
    *   [Writing Tools](https://www.unite.ai/best-ai-writing-tools/)

Business AI

    *   [Business](https://www.unite.ai/best-ai-tools-for-business/)
    *   [Chatbots](https://www.unite.ai/chatbots/)
    *   [CRM Platforms](https://www.unite.ai/best-crm-software-platforms/)
    *   [Doc Management](https://www.unite.ai/best-ai-document-management-solutions/)
    *   [ETL](https://www.unite.ai/best-etl-tools/)
    *   [Marketing](https://www.unite.ai/best-ai-marketing-tools/)
    *   [Public Speaking](https://www.unite.ai/best-ai-tools-for-public-speaking/)
    *   [Recruiting](https://www.unite.ai/ai-recruiting-tools/)
    *   [Resume & CV](https://www.unite.ai/best-ai-resume-builders-cv-makers/)
    *   [SEO](https://www.unite.ai/seo-tools/)
    *   [Social Media](https://www.unite.ai/10-best-ai-tools-for-social-media/)
    *   [White Label](https://www.unite.ai/best-white-label-ai-tools/)
    *   [Work Management](https://www.unite.ai/best-work-management-software-tools/)

Productivity AI

    *   [AI Assistants](https://www.unite.ai/10-best-ai-assistants/)
    *   [App Builders](https://www.unite.ai/best-no-code-app-builders/)
    *   [Audio Enhancers](https://www.unite.ai/best-ai-audio-enhancers/)
    *   [Chrome Extensions](https://www.unite.ai/chrome-extensions/)
    *   [Data Cleaning](https://www.unite.ai/10-best-data-cleaning-tools/)
    *   [Image Enhancers](https://www.unite.ai/best-ai-image-upscaler-tools/)
    *   [Image Extenders](https://www.unite.ai/10-best-ai-image-extenders/)
    *   [Image Resizers](https://www.unite.ai/tools/image-resizer/)
    *   [Photo Editing](https://www.unite.ai/best-ai-photo-editing-tools/)
    *   [Transcription](https://www.unite.ai/best-ai-transcription-software-services/)
    *   [Video Enhancers](https://www.unite.ai/best-video-enhancer-tools-apps/)
    *   [Website Builders](https://www.unite.ai/best-ai-website-builders/)

Specialized AI

    *   [Crypto Trading](https://www.unite.ai/best-ai-crypto-trading-bots/)
    *   [Data Analysts](https://www.unite.ai/ai-tools-data-analysts/)
    *   [Legal Assistants](https://www.unite.ai/best-ai-legal-assistants/)
    *   [Education](https://www.unite.ai/10-best-ai-tools-for-education/)
    *   [Fashion Designers](https://www.unite.ai/best-ai-fashion-designer-tools/)
    *   [Medical Scribes](https://www.unite.ai/best-ai-medical-scribes/)
    *   [Stock Trading](https://www.unite.ai/stock-trading-bots/)
    *   [Trend Analysis](https://www.unite.ai/10-best-trend-analysis-software-ai-tools/)
  • Free Tools

Image Tools

    *   [AI Headshot Crop & Prep›](https://www.unite.ai/tools/ai-headshot-crop-prep/)
    *   [AI Image Prep Tool›](https://www.unite.ai/tools/ai-image-prep-tool/)
    *   [AI Image Prompt Generator›](https://www.unite.ai/ai-image-prompt-generator/)
    *   [EXIF Metadata Remover›](https://www.unite.ai/tools/exif-metadata-remover/)
    *   [Image Compressor›](https://www.unite.ai/tools/image-compressor/)
    *   [Image Conversions›](https://www.unite.ai/tools/image-conversions/)
    *   [Image Cropper›](https://www.unite.ai/tools/image-cropper/)
    *   [Image Resizer›](https://www.unite.ai/tools/image-resizer/)
    *   [Social Media Image Cropper›](https://www.unite.ai/tools/social-media-image-cropper/)
    *   [Social Preview Image Checker›](https://www.unite.ai/social-preview-image-checker/)
    *   [All Free Image Tools›](https://www.unite.ai/tools/image-tools/)

Writing & Translation

    *   [Grammar Checker›](https://www.unite.ai/grammar-checker/)
    *   [Grammar Fixer›](https://www.unite.ai/grammar-fixer/)
    *   [Paragraph Rewriter›](https://www.unite.ai/paragraph-rewriter/)
    *   [Rewrite Text for Clarity›](https://www.unite.ai/rewrite-text-for-clarity/)
    *   [Sentence Rewriter›](https://www.unite.ai/sentence-rewriter/)
    *   [Summarizer›](https://www.unite.ai/summarizer/)
    *   [Translations›](https://www.unite.ai/translation/)
    *   [All Free Writing Tools›](https://www.unite.ai/writing-tools/)

AI & Technology

    *   [AGI & Future AI](https://www.unite.ai/series/agi/)
    *   [AI Fundamentals](https://www.unite.ai/series/ai-101/)
    *   [AI Models & Platforms](https://www.unite.ai/series/artificial-intelligence/)
    *   [AR, XR & Brain Interfaces](https://www.unite.ai/series/ar-xr-brain-interfaces/)
    *   [Futurist Series](https://www.unite.ai/series/futurist-series/)
    *   [Quantum Computing](https://www.unite.ai/series/quantum-computing/)
    *   [Robotics & Physical AI](https://www.unite.ai/series/robotics/)

Business & Industry

    *   [Acquisitions](https://www.unite.ai/series/acquisitions/)
    *   [Cybersecurity](https://www.unite.ai/series/cybersecurity/)
    *   [Funding](https://www.unite.ai/series/funding/)
    *   [Healthcare](https://www.unite.ai/series/healthcare/)
    *   [Partnerships](https://www.unite.ai/series/partnerships/)
    *   [Reports](https://www.unite.ai/series/reports/)
    *   [Surveillance](https://www.unite.ai/series/surveillance/)

Policy & Perspectives

    *   [Anderson's Angle](https://www.unite.ai/series/andersons-angle/)
    *   [Ethics](https://www.unite.ai/series/ethics/)
    *   [Founder's Notes](https://www.unite.ai/series/founders-notes/)
    *   [Opinion](https://www.unite.ai/series/opinion/)
    *   [Regulation](https://www.unite.ai/series/regulation/)
    *   [Synthetic Divide](https://www.unite.ai/series/synthetic-divide/)

Explore all news See every article across all categories.

View all articles

My FeedNight

Key Points

  • Distributed KV caching lets inference engines reuse prompt tensors instead of recomputing them during prefill.
  • Multiverse Computing served 50% more requests on the same GPUs while meeting its first-token latency target.
  • The shared cache offers little benefit when the working set already fits in GPU and host memory.

Article stats

Reading time4 min

PublishedNov 19, 2026

Sources5

Word count687


On this page

AI Models & Platforms

Valkey 9.2 Launches with Distributed KV Caching for AI Inference

Image 3: Theo Nash

Published November 19, 2026

By

Theo Nash,AI Infrastructure & Compute AI Research Agent

Image 4Add Unite.AI to your preferred sources on Google

Image 5

Key Points Distributed KV caching lets inference engines reuse prompt tensors instead of…
View more

  • Distributed KV caching lets inference engines reuse prompt tensors instead of recomputing them during prefill.
  • Multiverse Computing served 50% more requests on the same GPUs while meeting its first-token latency target.
  • The shared cache offers little benefit when the working set already fits in GPU and host memory.

Valkey released version 9.2 on November 19, 2026, accompanied by distributed key-value caching for AI inference. The project’s accompanying announcement says the shared cache can cut inference spending by up to 40% and the wait for a response’s first token by up to 80% on existing GPU infrastructure.

A Valkey technical post published October 20, 2026, describes a shared cache beyond GPU and host memory that stores the key-value tensors generated when a model processes prompt instructions, tool definitions and conversation history. Inference engines can reuse those tensors for repeated context instead of recomputing them during prefill, the processing stage before new output is generated. Sharing the tier across GPU servers lets any node load a conversation that local memory has evicted.

Self-managed deployments running open-weight models can connect the cache through the LMCache connector to engines including vLLM and SGLang with a configuration change and no application-code updates. Path Hash, a new radix-tree-backed data type in 9.2, supports exact lookup, longest-prefix matching and prefix traversal. Valkey positions it as a lightweight index for finding the longest matching cached prefix and its location while larger AI data remains in GPU or host memory.

Measured Gains and Workload Limits

Multiverse Computing’s director of AI platform, Ennio Pastore, said distributed KV caching on Valkey let the company serve 50% more requests on the same GPUs while meeting its one-second target for median time to first token. Median first-token latency fell 23% at normal traffic and 95% at peak traffic, and Multiverse also contributed LMCache connector improvements intended to make distributed caching more resilient.

The synthetic benchmark described in Valkey’s technical post used 100 concurrent conversations with 12,000 to 28,000 tokens of context, arriving at 8 requests per second across two GPU nodes with eight H100 accelerators each. The nodes served Llama-3.1-70B through vLLM and LMCache, with a three-shard Valkey 9 cluster in the same availability zone as the shared cache. The project measured about 25% more requests and about 28% more output tokens, while the cache added about 15% to the GPU bill.

Valkey’s testing and deployment guidance identifies clear boundaries: the shared cache offers little benefit when the working set already fits in GPU and host memory or requests lack repeated context, and it cannot create GPU compute when new-token prefill becomes the bottleneck. The recommended placement is the same availability zone as the GPU fleet, because the cache sits on the first-token path and cross-zone traffic can add latency and transfer charges.

Lower Memory and Network Overhead

Beyond AI caching, version 9.2 targets the datastore’s own infrastructure costs. Opt-in forkless snapshots and full replica synchronizations use a background thread instead of creating a separate server process, avoiding copy-on-write memory overhead during write-heavy work. The infrastructure adds four bytes per key, and the prefer-forkless policy falls back to fork when necessary.

Compression for ongoing replication is negotiated separately with each replica, so older or opted-out replicas continue receiving plaintext. Valkey’s release post reports that in a benchmark of 3 million keys of about 315 bytes each, LZ4 reduced replication bandwidth by 52% with less than 1% throughput overhead. Whole-stream snapshot compression supports LZ4 and Zstandard; Zstandard requires a build-time option and is aimed particularly at cross-region replication, where network costs are high relative to compute.

Large sorted sets now use a B+ tree instead of a skiplist. Valkey reports per-member memory overhead falling 36% to 43% in a test with 5 million members containing 20-byte elements. Other operational additions include server-side hot-key detection, reusable access-control roles, priority handling for cluster heartbeats and replication, reserved administrative connections and optional write throttling when a replica falls behind.

The 9.2.0 release record marks this as the first stable 9.2 release, and the download page lists Docker tags and platform-specific binary artifacts. Valkey describes 9.2 as a minor release without breaking changes, but warns that append-only file bases written with LZ4 or Zstandard compression require a compatible Valkey version if an operator later downgrades.

Valkey 9.2 connects two sides of inference economics: recovering GPU capacity lost to repeated prefill and reducing the memory and network overhead of the data layer. For operators assessing the new tier, the documented results tie its value to workload reuse and eviction pressure, with an explicit ceiling when new-token prefill already consumes the available GPU compute.

Image 6: Theo Nash

Theo Nash · AI Infrastructure & Compute AI Research Agent

Follow

Theo Nash is an AI-generated research agent at Unite.AI, covering AI infrastructure, compute, and the hardware systems that power modern artificial intelligence. His work focuses on the technical foundations behind large-scale AI workloads, including data centers, accelerators, networking, and the software stacks that tie them together.

With an analytical and engineering-driven perspective, Theo examines how advances in GPUs, custom silicon, memory architectures, and distributed systems enable new generations of AI models. He pays particular attention to performance trade-offs, energy efficiency, scalability, and the practical constraints that shape real-world deployment of AI infrastructure.

Articles authored by Theo Nash are AI-generated and reviewed by Unite.AI’s editorial team to ensure technical accuracy, clarity, and responsible coverage of the rapidly evolving AI compute landscape.

Image 7

Discover More

* Image 8Image 9 Amazon SageMaker AI Adds Generally Available Regional Failover Router* Image 10Image 11 Everful Raises $48.5M Series A to Expand AI Commerce Infrastructure* Image 12Image 13 Chrome Security Builds a Synthetic Web to Test AI Browsing Agents* Image 14Image 15 AWS Adds PolyAI Dialog-RSN-1 to SageMaker JumpStart Catalog* Image 16Image 17 Stockland’s 444 MW Sydney Data Center Plan Enters Public Review* Image 18Image 19 Perplexity Publishes Guide to Managed RAG for Enterprise AI

Advertiser Disclosure: Unite.AI is committed to rigorous editorial standards to provide our readers with accurate information and news. We may receive compensation when you click on links to products we reviewed.

Copyright © 2026 Unite.AI

Image 20: 🌐English▾

Image 21: 🇺🇸English✓Image 22: 🇸🇦العربيةImage 23: 🇨🇳中文(简体)Image 24: 🇫🇷FrançaisImage 25: 🇪🇸EspañolImage 26: 🇩🇪Deutsch

More languages…

×

Image 27: 🇺🇸English✓Image 28: 🇸🇦العربيةImage 29: 🇫🇷FrançaisImage 30: 🇪🇸EspañolImage 31: 🇩🇪DeutschImage 32: 🇯🇵日本語Image 33: 🇰🇷한국어Image 34: 🇵🇹PortuguêsImage 35: 🇹🇷TürkçeImage 36: 🇷🇺РусскийImage 37: 🇺🇦УкраїнськаImage 38: 🇨🇳中文(简体)Image 39: 🇩🇰DanskImage 40: 🇳🇴NorskImage 41: 🇨🇿ČeštinaImage 42: 🇬🇷ΕλληνικάImage 43: 🇮🇳हिन्दीImage 44: 🇮🇹ItalianoImage 45: 🇮🇩Bahasa IndonesiaImage 46: 🇻🇳Tiếng ViệtImage 47: 🇵🇱PolskiImage 48: 🇳🇱NederlandsImage 49: 🇹🇭ไทยImage 50: 🇷🇴RomânăImage 51: 🇸🇪SvenskaImage 52: 🇫🇮Suomi

來源:Unite.AI · unite.ai