IBM 將 Spyre 加速器整合為 PyTorch 一級後端

IBM said in an IBM Research blog post published October 26, 2026 that software updates delivered across the PyTorch ecosystem in the second half of 2026 have made its Spyre dataflow accelerator a first-class PyTorch backend, delivering on a plan IBM and the PyTorch Foundation outlined in October 2025.

IBM describes Spyre as its dataflow AI accelerator, optimized for inference and designed for enterprise teams running AI alongside their applications and data on IBM Z, LinuxONE, and Power systems. Its reduced-precision compute targets the matrix-heavy work in language generation and embedding models, and unlike a conventional GPU, which executes kernels as instruction streams across scheduled groups of threads, Spyre triggers computation as data arrives at its compute engines.

In the second half of 2026, IBM, Red Hat, and collaborators in the PyTorch Foundation including Meta and Hugging Face pushed updates covering day-one model enablement, hardware-agnostic models in vLLM, continuous regression and release testing, and native-device integration, the post states. At PyTorch Conference 2025, in an IBM Research post dated October 21, 2025, IBM said it planned deeper Spyre enablement for 2026, with Spyre participating as a first-class backend in open-source inference workflows and a focus on unified compiler pathways, optimized kernel generation, and transparent multi-accelerator scheduling. IBM had by then adopted vLLM as its inference runtime and torch.compile as the frontend compiler for integrating accelerators such as Spyre, and it had contributed a Spyre plugin for vLLM that enables paged attention for large language model inference.

A Compiler Rebuilt on Inductor

At the center of the integration is torch-spyre, a PyTorch backend that compiles ordinary PyTorch code into an executable plan for Spyre. Most of a model’s logic passes through the PyTorch compiler unchanged; when an operation cannot yet be lowered effectively, an adapter can replace it with a mathematically equivalent form that the compiler can handle.

In a PyTorch Foundation post published October 13, 2026, Dave Grove, Olivier Tardieu, and the IBM Spyre Team described moving torch-spyre’s torch.compile backend from sendnn, a compilation stack built entirely from scratch, to backend=inductor, an open-source backend built by extending PyTorch’s own Inductor. Measured with the lizard code analyzer, sendnn totals roughly 273,000 lines of C++ and Python across 6,170 functions, while the Inductor extension comes in at roughly 38,600 lines of Python across 1,878 functions — about 7x fewer lines with lower per-function complexity, the authors reported.

Spyre is a 75-watt, PCIe-attached accelerator built from 32 small cores wired together over a ring, each with its own 2 MB scratchpad rather than a shared cache, and it moves data in fixed 128-byte chunks called sticks. The compiler work concentrated on stick-granularity data movement, scratchpad planning, and compile-time core assignment, the post states.

Reported measurements include a 29–69% reduction in matmul kernel time from tiling and layout choices, a drop in softmax median kernel time from 32.51 microseconds to 23.67 microseconds — a 27% reduction — from scratchpad-aware scheduling, and a fused MLP-style kernel that lifted core utilization from roughly two-thirds to about 80% while cutting kernel time around 17%. On a synthetic graph built to stress those decisions, a new on-chip relayout replaced an 80-microsecond device-memory round trip with a 1.1-microsecond shuffle between cores. The post also notes that torch-spyre’s need to register device-specific decompositions helped drive improvements to out-of-tree Inductor backend support that landed upstream in PyTorch 2.12 as two merged pull requests.

Native-Device Integration

Spyre becomes a native PyTorch device through PrivateUse1 registration, which gives tensor.to(“spyre”) a real device identity: PyTorch allocator storage keeps tensors resident on the device, streams provide ordering semantics, and runtime events connect dependent work across streams. The card carries 32 cores connected by a high-bandwidth ring, each with 2 MB of local scratchpad, and up to 128 GB of LPDDR5 for tensors and programs, and the current runtime patches programs at launch, substituting symbolic tensor addresses with real ones in three ordered steps.

In a demonstration on Granite 3.3 8B at batch size 1 and sequence length 1024, moving graph construction off the data-transfer path improved prefill by 9.9% and decode by 13.6%, the team reported. Removing that work from compute dispatch as well brought totals to 1.7x faster prefill and 2.4x faster decode, with roughly three quarters of the gain coming from the compute path.

“The win is that neither Spyre nor PyTorch has to pretend to be something else,” the authors wrote in Building Spyre as a Native PyTorch Device, published October 8, 2026, adding that Spyre keeps its dataflow execution model while users get a native PyTorch device with familiar tensor placement, streams, and compilation.

vLLM Portability and the Llm-D Connection

Spyre is an out-of-tree accelerator rather than one of the GPU platforms most commonly supported by AI software, relying on TorchDynamo to trace a model graph and TorchInductor to lower that graph for the target hardware. Within vLLM, extension mechanisms such as CustomOp and PluggableLayer allow a hardware plugin to customize behavior without reimplementing large portions of the software stack.

In Hardware-Agnostic Models in vLLM, published September 22, 2026 by authors from IBM, Meta, and Hugging Face, the team described vLLM’s shift toward hardware-specific “flat” model definitions and introduced hardware-agnostic layers built on four design principles: compilable, extensible, isolated from hardware-specific paths, and portable. Early support has landed in vLLM’s main branch and can be enabled with USE_HW_AGNOSTIC=1 on the transformers backend. The pathway has been validated with the Spyre out-of-tree plugin on models including Gemma 4, Qwen3, and Granite 4.2, and on NVIDIA H100 GPUs the hardware-agnostic layers delivered total token throughput within 3.4% of the native implementation, measured as the geometric mean across three recent models, the authors reported.

Spyre’s PyTorch integration is also how it connects to llm-d, the IBM post explains: llm-d runs distributed inference on top of vLLM across multiple accelerator types, vLLM reaches each accelerator through PyTorch, and once a device has a working PyTorch backend, a vLLM hardware plugin can expose it so llm-d can schedule work onto it.

Day-One Enablement and Continuous Testing

An August 20, 2026 post from the IBM Spyre Team describes HF-adapters, a project that uses coding agents to draft runtime patches letting stock Hugging Face Transformers models run on Spyre. From mid-April to late June 2026, 13 distinct adapters came to cover 7,960 of the 10,000 most-downloaded Hugging Face embedding models, of which 6,804 pass their end-to-end test on Spyre, the post reports. One example replaces the gelu_new activation’s torch.pow(x, 3.0) with the mathematically identical x x x, which the torch-spyre stack can lower; humans retain the diagnosis work, localizing device-only failures that arise from fused operations and checking each agent’s conclusions against end-to-end results.

In a September 30, 2026 post, IBM and Red Hat engineers described torch-spyre’s L2 integration with PyTorch’s Cross-Repository CI Relay, which triggers downstream testing on upstream PyTorch changes and reports results back to PyTorch’s CI dashboard. The integration pairs an agentic four-stage test-selection pipeline with a declarative YAML test-reuse framework, and upgrading from PyTorch 2.13 to 2.14 meant evaluating roughly 4,000 changed tests instead of tens of thousands, the authors wrote, adding that they intend to work with the PyTorch team to upstream the reusable parts with PyTorch OpenReg as the reference point.

For IBM Z, LinuxONE, and Power11 users, the intended experience is to obtain models from ecosystems like Hugging Face and use substantially the same PyTorch-based workflows they would use with Nvidia GPUs, with shared PyTorch extension points hiding much of the architectural difference, the IBM post states. The authors of the vLLM post said they plan to begin including hardware-agnostic models in CI and gradually switch to that pathway as the default for serving models on Spyre.