Eclypsium 研究展示遭入侵硬件如何在推理期間改變 AI 模型

Eclypsium has demonstrated how compromised computing infrastructure can change an AI model’s behavior during inference while its original files remain intact. The research, published October 29 in Trusted AI Requires Trusted Infrastructure, shows why checking a model’s signature or file hash does not establish the integrity of the model executing in memory.

In a controlled demonstration supported by NVIDIA, researchers used a compromised data processing unit, or DPU, together with permissive memory isolation to modify runtime model weights. The model’s answers changed without modifying the stored model or inference service. The finding places hardware, firmware, and device configuration inside the AI security boundary.

A Different Route to Changing an AI Model

AI security discussions often focus on malicious prompts, poisoned training data, and unsafe tool use. This research examines a lower layer: the infrastructure that holds and moves a model’s parameters while it runs.

Weights are the numerical parameters that shape a model’s responses. A valid file on disk can be loaded into memory and subsequently changed by a sufficiently privileged component. A later check of the original file still sees the expected bytes, even though an answer was generated using altered runtime state.

Eclypsium’s demonstration visibly changed analytical output on a geopolitical question. The broader concern is that runtime manipulation could affect other sensitive uses, including generated code and automated decisions. Those possibilities should be distinguished from the specific behavior shown in the laboratory.

The claim of remaining undetected also has a defined scope: the stored-model integrity checks used in the demonstration did not reveal the memory changes. That does not establish invisibility to every form of monitoring or forensic examination.

The Attack Conditions Matter

The test used a NVIDIA Grace Hopper environment. Its direct-memory-access path depended on a compromised privileged component and configuration settings that allowed access to sensitive memory. The researchers identify IOMMU passthrough and permissive PCIe isolation as important conditions, explicitly noting that these were not the default Linux settings.

An IOMMU helps control which memory a peripheral can reach. In the stronger isolation configuration tested by the researchers, the same direct path was unavailable. The result therefore underscores the importance of verifying actual deployment settings, rather than assuming that an installed security capability is actively enforcing a boundary.

A DPU is a programmable infrastructure processor used for networking, storage, and security services. Its ability to move data efficiently is valuable, but privileges that serve legitimate infrastructure functions can become dangerous if an attacker gains control of the component.

NVIDIA’s Response: Verify the Trust Boundaries

In its account of the collaboration, NVIDIA describes BlueField’s authenticated boot, hardware-rooted trust, isolation, firmware-update protections, and attestation. It also emphasizes that cloud operators choose different security configurations for different trust models.

The companies’ message is that operators need evidence about those choices. Memory-access controls, device isolation, and management interfaces must be assessed in the context of a deployment’s threat model. A feature’s availability alone does not show that it is correctly configured.

NVIDIA and Eclypsium also describe the issue as extending beyond a single vendor. AI systems depend on privileged host software, accelerator firmware, management controllers, and data-transfer mechanisms. The exact paths differ by architecture; the need to account for those components does not.

Why Agent Security Also Depends on Infrastructure Security

The stakes grow when a model can call tools, access enterprise data, or initiate actions. Infrastructure tampering can undermine assumptions made by application-level safeguards about which model is actually running.

NVIDIA’s Open Agent Safety Platform illustrates the layered response. OpenShell provides runtime governance, while NVIDIA Sentry on BlueField operates from a hardware-isolated security domain outside the host and agent. The architecture aims to monitor behavior and enforce policy from a separate control point.

That independent position creates another security responsibility: the enforcement hardware and its software stack must themselves remain trustworthy. Moving a control outside the application reduces one dependency while increasing the importance of verifying the component entrusted with that control.

From Hardware Inventory to Continuous Assurance

Eclypsium calls the disconnect between model-focused assurance and infrastructure integrity the “AI Infrastructure Assurance Gap.” Its infrastructure assurance platform inventories components, assesses firmware and configuration, compares device state with approved baselines, and monitors changes. The company positions those capabilities as complementary to existing application and model safeguards.

For operators, the practical work begins with treating DPUs, GPUs, management controllers, and their firmware as individually managed assets. Each needs ownership, version tracking, vulnerability assessment, and an understood update process. Management-plane access and third-party software on infrastructure processors belong in that review.

Checks should also follow the hardware through deployment, updates, incident recovery, and reassignment. A clean operating-system image does not by itself establish that every lower-level component is in an approved state. Shared infrastructure makes that lifecycle especially consequential because hardware may serve successive customers.

Checking the Running System, Not Just the Artifact

Model signatures remain useful: they answer an important question about the artifact being loaded. Secure boot, attestation, runtime isolation, and continuous monitoring answer different questions about the system that executes it. None should be treated as a substitute for all the others.

The significance of Eclypsium’s research is that a model can pass one integrity check while another part of its execution chain has changed. For organizations deploying AI into consequential workflows, infrastructure verification needs to accompany model evaluation throughout operation.