跳到正文
DEV Community· RuChating·· 59 分鐘前AI 評分55

SC26 後,AI 集羣容量規劃從 GPU 轉向機架、供電、冷卻與軟件版本

The rack is the new GPU: capacity planning after SC26

AI 導讀

SC26 後的系統發布顯示,AI 集羣容量規劃不能只按 GPU 數量估算,還須納入機架、供電、冷卻和軟件版本。文中列舉 800 V DC 電源機架、2.3 MW 冷卻液分配單元,以及可裝入既有機架的 Nova8 4U 八 GPU 系統,呈現新建與改造機房的不同路徑。SGLang v0.5.23 和 vLLM 0.34.0 的例子則説明,服務引擎、快取政策和軟件更新也會改變有效容量及延遲表現。

正文

For most of the last decade, sizing an AI cluster was a surprisingly simple exercise. You picked an accelerator, counted how many you could afford, multiplied by a utilisation figure and called the result your capacity. The building was somebody else's problem — a row of racks, some cables, a chiller that had always worked fine.

The announcements coming out of Supercomputing 2026 in Chicago make that habit obsolete. If you are responsible for an inference or training platform now, the unit you should be planning in is not the GPU. It is the rack, the power feed and the water loop — plus the exact software build that determines how much work you can extract from the silicon you already own.

What the vendors actually shipped

Look at the pattern in the post-show announcements rather than at any single one. On 18 November, Compal announced a rack-scale liquid-cooled system built on NVIDIA's HGX Rubin NVL8 platform. Giga Computing showed its own rack-scale infrastructure and liquid cooling at the same show. On 19 November, HPCWire collected new systems from Compal, Penguin Solutions, ASUS and Dell — rack-scale machines, liquid cooling and storage, all sold as integrated products rather than as components an integrator would assemble.

The most revealing number came from a company that sells none of the accelerators. On 16 November, nVent said its liquid-cooling portfolio has more than four gigawatts deployed, displayed a 2.3-megawatt modular coolant distribution unit, and unveiled an 800-volt DC power rack in its MAVERICK line.

Four gigawatts of deployed cooling is not a roadmap. It is installed hardware. When a cooling vendor can quote installed capacity in the same units as a utility, the thermal problem has moved from the research budget into the capital plan.

The second number worth staring at is 800 volts. Moving distribution to 800 V DC removes conversion stages between the utility feed and the accelerator: fewer transformers, fewer UPS blocks, fewer rectifiers, less copper and less heat in the power room. The same logic that pushed the industry from 12 V to 48 V inside a server now applies to the whole hall. This is not an exotic Chinese experiment any more; it is a SKU in a vendor catalogue.

The retrofit market is the interesting part

Not everyone gets to build a new hall. Most teams inherit an existing room with a finite power allowance, a ceiling height designed for 1U servers and no water anywhere near the rows. That is where the more pragmatic announcements landed.

Nexalus and Renda launched the Nova8: eight NVIDIA Blackwell GPUs in a single 4U chassis, with CPU, memory, storage, power and cooling integrated. You do not rebuild the room around it. You slide it into the rack you already have.

This is a genuinely different product category from the 100 kW rack. One is a greenfield datacenter designed around a liquid loop; the other is an upgrade path for the enormous installed base of air-cooled halls. If your organization has ever been told "we cannot deploy GPUs here because the building cannot take them", the second category is the one that matters. Capacity planning starts with an inventory of what the building can actually accept, not with a GPU datasheet.

There is a third path too. Fujitsu began global sales of its 144-core MONAKA processor and MONAKA Server in November, built from a 2 nm core die stacked with 5 nm SRAM and I/O dies. It was shown at SC26 as a sovereign-AI server, but the engineering detail that matters for a small team is that it targets inference in air-cooled facilities. A fast CPU in an existing hall can beat an accelerator that requires a construction project. That is not a benchmark claim; it is a scheduling and facilities claim.

Software is part of the capacity number

Here is the part that most capacity models still get wrong. The amount of work you get from a rack is not a property of the rack. It is a joint property of the hardware and the exact software build running on it.

The SGLang v0.5.23 release (2 November, 1,007 pull requests from 257 contributors) is a good example. Its Paged Experts feature can run a mixture-of-experts model larger than a single GPU's memory on one consumer card, at up to 30 times the speed of CPU offload. It does this by paging expert weights the way an operating system pages memory — extending a technique that used to apply only to KV caches. The same release added fused persistent "megakernel" paths for NVIDIA and AMD that make single-user decode up to 44% faster on a GB300 and 75% faster on an MI350X.

The vLLM 0.34.0 release tells the same story from the operations side. Extensible KV cache sizing grew a MiniMax-M3 cache on an H200 from 433k to 672k tokens, but at concurrency 128 some models pay 6–18% higher per-token decode latency. That is a capacity decision disguised as a release note: more concurrent context, slightly slower individual responses. The release also added checkpoint/restore through CRIU and the ability to drop a failed rank from a data-parallel/expert-parallel group and redistribute its experts without restarting the service.

If your model of a GPU is "N tokens per second, forever", it will be wrong in both directions. A software upgrade can multiply effective capacity; a new default can quietly trade tail latency for throughput. Capacity is a versioned number now, and it needs a change log.

A capacity checklist that matches the new reality

If you are rewriting a platform plan this quarter, these are the questions that produce useful answers:

  1. What does the hall accept today? Available kilowatts per rack, ceiling height, water or lack of it, and the actual power feed. A 4U eight-GPU server may be the only option that fits.
  2. Which distribution voltage are you buying into? An 800 V DC rack changes the power room, not just the PDU. Decide before ordering, because retrofitting the electrical chain later is expensive.
  3. What is the cooling unit, and who owns it? A 2.3 MW CDU is a facility asset with a vendor and an SLA. Treat it like one, with spares and a maintenance contract.
  4. Which software build is in the capacity number? Record the serving-engine version, the attention backend and the cache policy alongside tokens per second. Re-measure after every upgrade rather than assuming monotonic improvement.
  5. Does the service survive a dead rank? Checkpoint/restore and expert redistribution are the difference between a platform and a demo. Test them under an eight-hour load, not a five-minute smoke test.
  6. Is a CPU path viable for the low-traffic workloads? Air-cooled inference on a dense server can remove an entire class of capacity constraint for local or sovereign deployments.

The shift worth naming

The industry spent years treating the accelerator as the bottleneck and everything else as plumbing. The post-SC26 announcements say the opposite: the silicon is available in more forms than before, while the hall, the power feed and the software build decide whether you can use it.

That changes what "we need more GPUs" means. Sometimes it means buying a rack. Sometimes it means a CDU and an 800 V feed. Sometimes it means a serving-engine upgrade that makes the existing GPUs behave like new ones. The common thread is that the unit of measurement has moved up the stack — from the chip to the system that keeps the chip busy.

Engineers who keep planning in GPUs will keep producing spreadsheets that look precise and fail on commissioning day. The ones who plan in racks, watts and versioned software will be the ones whose clusters actually turn on.

來源:DEV Community · dev.to