x402 測試可能在從未讀取付款的伺服器上通過

Start with the thesis: Satisfied Is Not Established.

Payment-protocol test suites are usually written while the protocol is still moving. The protocol ships, its wire format settles, and the suite keeps speaking the draft. Every check still runs. Every check still returns a verdict.

The way to find out is to point the suite at a server that cannot be paid, and count the passes.

The measurement

The Agent Security Harness carries 54 tests for x402, the HTTP 402 payment protocol for agents. This week I ran them against a minimal server that follows the published x402 v2 HTTP transport: every request without a payment gets a 402 and a base64 PAYMENT-REQUIRED header describing how to pay. The server has no facilitator. It accepts no payment, ever.

The result: 43 of 54 PASS, 11 FAIL.

Neither number was evidence. X4-045 reported "Replay correctly handled (status 402, 402)". X4-046 reported "At most 1 settlement accepted ([402, 402]) — safe". The server never read a payment, so there was nothing to replay and nothing to settle.

Why

The probes were written against a pre-spec header format. They read the challenge from X-Payment-Amount and X-Payment-Recipient, and they send a forged payment as X-Payment-Authorization.

The published transport does neither. Version 1 puts the challenge in a JSON body and takes the payment in X-PAYMENT (spec). Version 2 puts it in a PAYMENT-REQUIRED header and takes the payment in PAYMENT-SIGNATURE (spec).

A spec server ignores the forged header and answers every probe with the 402 it sends to everyone. The test sees a non-200 status and records the attack as rejected.

A 402 the server would have sent anyway is not a rejection.

The failures were wrong in the mirror direction. X4-001 failed the server for missing x-payment-* headers, which the spec does not have.

Why the usual guards stay quiet

The harness already refuses a verdict when the target never answers: a closed port, a host returning 404 to everything, an empty 2xx. Each of those shapes has a guard and a test that holds it.

This shape passes all of them. The target answered, correctly, in a protocol the probes did not speak. Any guard that asks "did the target respond?" stays quiet, because it did. The only instrument that catches it is a target that follows the spec and cannot be paid.

The fix

An outside review flagged the format mismatch. The measurement and the fix shipped the same day.

PR #672 teaches the module to recognise a spec challenge. Against a spec target, five tests keep a verdict, because their evidence is the server's own response: the challenge is well-formed, the payee is stable, two leak scans, and behaviour under load. The other 49 report INCONCLUSIVE and say why.

Against the same server, the run now reads 5 PASS, 49 INCONCLUSIVE. A smaller number, and one a reader can rely on.

A guard test runs the suite against spec v1 and v2 targets and fails if any of the 49 returns a verdict. It also checks that X4-001 can still fail a spec challenge with no payee.

The next step is probes that speak the spec. Each needs a correctly signed payment the server must accept, as the positive control, before any rejection means anything.

Run the same check on your suite

Stand up a server that returns the spec challenge and accepts nothing. Run your payment tests against it. Every PASS on a test that claims to exercise a payment is a test that never reached one.

Michael K. Saleme maintains the Agent Security Harness. The counts come from runs on 30 October 2026: before the fix against harness main at 4d7528c, after it against the fix merged as 76ef629. The guard is testing/test_x402_spec_target.py, and the CHANGELOG entry records the correction.