跳到正文
SiliconANGLE· Ryan Stevens·· 4 小時前AI 評分31

Liqid 推動可按新模型調整配置的 AI 推理機櫃

AI’s next build-out: Inference racks that resize themselves for every new model

AI 導讀

Liqid 提出在機櫃層級池化 GPU 和記憶體,按模型或工作負載動態分配資源予伺服器,取代固定的 GPU 與記憶體比例。公司在 SC26 宣佈推出全代管、單租戶服務 Liqid as a Service,並正轉向預先配置的設備。

正文

AI’s next build-out: Inference racks that resize themselves for every new model

Memory pooling is fast becoming one of the sharpest tools in AI inference economics, with companies racing to wring more output from every graphics processing unit.

Most AI servers still ship with a fixed GPU-to-memory ratio that holds for the life of the machine. But that is a poor fit when nobody knows what model they will deploy six months from now, according to Sumit Puri (pictured), co-founder and chief technical officer of Liqid Inc.

“Our core fundamental technology is around pooling and sharing the most critical resources in the data center, which are now GPU and memory,” Puri said. “Rather than statically deploying these things inside of a server, our vision is simple. Let’s put pools of resources in the rack, GPUs and memory, and dynamically assign them to any server that needs it based upon the model or the workload that we are looking to deploy.”

Puri spoke with theCUBE Research’s John Furrier and Gemma Allen at SC26, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed pooling GPUs and memory, the tokenomics of AI inference and Liqid as a Service. (* Disclosure below.)

Memory pooling meets the tokenomics test

Liqid measures payoff in tokenomics, with the he yardsticks being tokens per second, tokens per watt and, above all, tokens per dollar. Supply is another pressure point in the AI infrastructure build-out, with memory among its most constrained components. But delivering only the memory each application needs stretches scarce supply further, Puri explained.

“Supply chain is a massive constraint for all of us — memory being a giantly massive constrained thing,” he said. “If we can do things like pool memory, share memory, optimally deliver memory to the server in the right amount of memory as required by the application, we can directly address some of those supply chain issues.”

Liqid is also widening how customers buy memory pooling. At SC26, the company announced Liqid as a Service, a fully managed, single-tenant offering, and it is moving toward preconfigured appliances. The bet is that most enterprises will deploy models rather than build them and therefore require their own capacity, according to Puri.

“The world went to unlimited consumption of minutes, and that’s where we believe tokens are headed,” Puri said. “Everybody’s going to own their own token machine so they can consume these tokens at will.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of SC26:

(* Disclosure: Liqid sponsored this segment of theCUBE. Neither Liqid nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE

A message from John Furrier, co-founder of SiliconANGLE:

Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.

  • 15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more
  • 11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network

SiliconANGLE Media is a recognized leader in digital media innovation, uniting breakthrough technology, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — with flagship locations in Silicon Valley and the New York Stock Exchange — SiliconANGLE Media operates at the intersection of media, technology and AI.

Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.

來源:SiliconANGLE · siliconangle.com