跳到正文
SiliconANGLE· Jonathan Anthony·· 3 天前AI 評分42

數據中心建設受阻,推理晶片轉往電訊及託管設施尋找空間

Stalled data center buildouts send inference chips hunting for space in telco and colo sites

AI 導讀

推理需求增長快於新 AI 數據中心建設,令晶片商轉往現有電訊及託管設施尋找空間。SambaNova 第五代晶片 SN50 針對推理解碼階段,採用風冷設計,適合安裝在現有設施。SambaNova 正展示以 Intel D-Series GPU 處理預填充、配合 SambaRack SN50 處理解碼的風冷異構配置。

正文

Stalled data center buildouts send inference chips hunting for space in telco and colo sites

Avatar photo

Demand for inference is outrunning the construction of new AI data centers, as power, capital and permitting delays push fresh capacity further out. That gap is sending chipmakers to hunt for room in existing facilities.

Many older telco and colocation sites have the space and power. But most can’t take today’s dense, liquid-cooled GPU racks without costly retrofits, which opens the door for air-cooled inference systems, according to Anton McGonnell (pictured), vice president of core product at SambaNova Systems Inc.

“We as an industry are just encumbered by the velocity of data center buildouts, and that’s been slowed down by just the physics of how long this takes, by the access to capital, and more and more by legislative hurdles — particularly in the U.S.,” McGonnell said.

McGonnell spoke with theCUBE Research’s John Furrier and Gemma Allen at SC26, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed AI data centers and the demand for air-cooled inference racks. (* Disclosure below.)

Reviving brownfield capacity for AI data centers

SambaNova’s fifth-generation chip, the SN50, targets the decode phase of inference, where output tokens are generated and data movement becomes the bottleneck. Its racks stay air-cooled and light enough to fit existing facilities, McGonnell explained. Newer GPU racks, by contrast, can demand reinforced flooring and 120 to 250 kilowatts of power per rack, he noted.

“On all of this stuff — the power and the cooling gear — there’s 18- to 24-month lead times to get this equipment into the data centers. Never mind all of the construction costs,” McGonnell said.

SambaNova, which investors valued at $11 billion in July, pairs its decode chips with partners that handle prefill, the compute-heavy first phase of inference. At SC26, it is demonstrating Intel’s D-Series GPUs for prefill alongside its SambaRack SN50 for decode in an air-cooled heterogeneous setup.

“They have developed a chip that has much, much better cost performance for the prefill phase of inference,” McGonnell said. “They’ve sacrificed HBM, which isn’t needed for prefill because it’s a compute-bound problem.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of SC26:

(* Disclosure: NYSE Wired sponsored this segment of theCUBE. Neither NYSE Wired nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE

A message from John Furrier, co-founder of SiliconANGLE:

Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.

  • 15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more
  • 11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network

SiliconANGLE Media is a recognized leader in digital media innovation, uniting breakthrough technology, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — with flagship locations in Silicon Valley and the New York Stock Exchange — SiliconANGLE Media operates at the intersection of media, technology and AI.

Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.

來源:SiliconANGLE · siliconangle.com