跳到正文
SiliconANGLE· Kelly Knight·· 2 小時前AI 評分42

D-Matrix押注堆疊式DRAM解決推理的記憶體頻寬問題

D-Matrix bets stacked DRAM can solve inference’s memory bandwidth problem

AI 導讀

D-Matrix押注以堆疊式3D DRAM提升推理記憶體頻寬,並將這項技術用於第二代晶片Raptor。D-Matrix表示,3D DRAM的能效和頻寬均為HBM的10倍,容量則介乎SRAM與HBM之間;Raptor預計明年推出。

正文

D-Matrix bets stacked DRAM can solve inference’s memory bandwidth problem

Avatar photo

As artificial intelligence workloads shift to inference, memory bandwidth decides whether hardware generates tokens quickly or sits idle. D-Matrix Corp. is answering with Raptor, a second-generation chip stacking 3D DRAM beneath its compute logic.

AI infrastructure has become a systems game, and winners combine memory, interconnect and software into cohesive designs, according to Satyam Srivastava (pictured), fellow at d-Matrix.

“It’s no longer a chip problem. Chip plays a part of it. Memory bandwidth is a fundamental advantage that has to be solved,” Srivastava said. “The next is energy efficiency. The next is interconnect efficiency. The next is software cohesiveness.”

Srivastava spoke with theCUBE Research’s John Furrier at SC26, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed d-Matrix’s Raptor chip and its stacked 3D DRAM, inference memory bandwidth and the shift toward heterogeneous computing. (* Disclosure below.)

Raptor brings a stacked-memory tier to inference

D-Matrix proves each risky technology on a test chip first. Jayhawk proved die-to-die connectivity, and Jayhawk II validated digital in-memory computing. The Corsair accelerator combined both and entered full production earlier this year, Srivastava explained.

“Fast forward, we have the Pavehawk chip, test chip, which is proving out 3D DRAM stacked underneath the logic die, which serves as the core technology for Raptor, which is our second-generation chip coming out next year,” Srivastava said.

Stacking keeps the distributed memory architecture intact but moves memory cells onto a separate layer rather than consuming logic die area. Data travels a short distance and never crosses an interposer. That puts 3D DRAM in a new tier between SRAM and high-bandwidth memory, Srivastava said.

“So you’re getting 10 times higher energy efficiency compared to HBM, 10 times higher bandwidth compared to HBM, where it sits in between SRAM and HBM in terms of capacity,” Srivastava said.

Pairing a Corsair-based server with an existing Hopper GPU fleet can raise utilization from about 20% to as much as 80%, meaning more tokens per dollar, Srivastava noted. D-Matrix has also adopted NVLink Fusion rackscale infrastructure from Nvidia Corp. to connect its accelerators to Nvidia’s AI factories.

“For us, it unlocks the opportunity to participate in a very established ecosystem, the world’s most popular AI factory. We get access to a very hardened design of an MGX rack,” Srivastava said.

The partnership points toward heterogeneous computing, where software orchestrates GPUs, CPUs and accelerators such as Raptor as one system, with the cluster as the unit of computing, Srivastava explained.

“So think of a chip, but that chip is an entire cluster and it has many different hardware units,” Srivastava said. “Some are GPUs, some are CPUs, some are ASIC accelerators, and you can use whichever one you like to serve the needs of the workload the best.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of SC26:

(* Disclosure: NYSE Wired sponsored this segment of theCUBE. Neither NYSE Wired nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE

A message from John Furrier, co-founder of SiliconANGLE:

Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.

  • 15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more
  • 11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network

SiliconANGLE Media is a recognized leader in digital media innovation, uniting breakthrough technology, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — with flagship locations in Silicon Valley and the New York Stock Exchange — SiliconANGLE Media operates at the intersection of media, technology and AI.

Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.

來源:SiliconANGLE · siliconangle.com