如何用 protocolprobe-selftest 在 CI 檢查自有 MCP 伺服器並在 GitHub 查看安全檢查結果

Most MCP security tooling reads a server's tool descriptions. That finds text that looks suspicious. It does not tell you what your server does when a client calls it.

This post is about a small command-line tool that does the second thing, for a server you run: it starts your server, connects as an MCP client, calls it, and reports what it observed. It is called protocolprobe-selftest, it is ISC-licensed, and the source is public: protocolprobe/protocolprobe-tools.

I build ProtocolProbe, a directory of dated facts about the servers in the official MCP registry. This tool is the part that runs on your machine, not ours. We use it on nobody else's server: you point it at your own, and the results stay on your computer.

Run it once

npx protocolprobe-selftest --i-own-this-server --stdio node ./dist/server.js

Everything after --stdio is your server's own command line, so a Python server works the same way (needs uv):

npx protocolprobe-selftest --i-own-this-server --stdio uv run --directory . server.py

The server is started in a temporary folder, so keep --directory . with uv run; a plain uv run server.py will not find your project.

--i-own-this-server is required on every run. It is a statement, not a technicality: the tool is for servers you own or are authorised to test.

What the output looks like

The result is a count over the checks that actually ran, with the rest listed, for example "N of M checks passed". Two rules are deliberate:

• No letter grade, no weighting. A single grade hides which check failed and why.
• Checks that could not run are listed, never counted as passed. If a check does not apply to your setup, or could not be tested, the output says so. A result where some checks did not run is marked partial.

Two of the checks are off unless you turn them on, because they are the kind that change data or send malformed input to your server:

npx protocolprobe-selftest --i-own-this-server --allow-mutating --allow-fuzzing --out report.json --stdio node ./dist/server.js

Use those against a development copy, not against a server holding data you care about.

Put it in CI

npx protocolprobe-selftest --i-own-this-server --fail-on-finding --sarif selftest.sarif --stdio node ./dist/server.js

• --fail-on-finding makes the job exit with code 5 when a check needs attention (VULNERABLE or FOUND). Without it the exit code is 0 whatever the result.
• --sarif <file> also writes the result as SARIF 2.1.0, which GitHub code scanning reads.

Only checks that need attention become SARIF results: VULNERABLE is an error, FOUND is a warning (a heuristic match for you to review). The score and the list of what did not run are kept in runs[0].properties.

A finding here is about the running server, not a line of code, so every result points at line 1 of one file. --sarif-anchor <file> picks that file; by default it is the first file in your --stdio command that is inside the current folder. With neither, the SARIF is valid but GitHub does not display the results, so run the command from your repository root.

One thing that is easy to get wrong in GitHub Actions: a step that exits 5 stops the steps after it. Give the upload step if: always(), or the SARIF is not uploaded in exactly the case where there are findings.

A workflow of that shape looks like this (adapt the build and server command to your project):

name: mcp-selftest
on: [push, pull_request]

jobs:
  selftest:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      security-events: write
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci && npm run build
      - run: npx protocolprobe-selftest --i-own-this-server --fail-on-finding --sarif selftest.sarif --stdio node ./dist/server.js
      - uses: github/codeql-action/upload-sarif@v3
        if: always()
        with:
          sarif_file: selftest.sarif

I have not run this exact workflow end to end for this post; the command line and the exit-code behaviour are from the tool's tests and documentation. If your first run behaves differently, open an issue with the tool version, your Node version and what you ran.

A server that is already deployed

From version 0.2.0 the tool can test your own Streamable HTTP server at a URL, but only after the server proves you control that exact URL:

1. Run the command with --url. It connects to nothing, prints a line such as protocolprobe-proof=pp1-..., and exits with code 3.
2. Put that line, on a line of its own, into your MCP server's instructions text and deploy.
3. Run the same command again within 30 minutes. It reads the instructions, and if the line is there it runs the checks on that same connection. The value is then used up.

The rules are enforced and tested: the value works once, only for the exact URL it was made for, expires after 30 minutes, and no redirect is followed. If your server needs a credential, you pass it through an environment variable (--bearer-env), never on the command line, and it is only sent to the proven URL and replaced with [redacted] in everything the tool prints or writes.

Be clear about what this proof is. It stops accidents and casual misuse: nobody can make someone else's server return a fresh random value. It is not enforcement. The tool is open source and could be changed to skip it, and other tools connect to any URL with no proof at all. Use --url only for a server you operate or are authorised to test.

What it cannot tell you

These limits are in the README too, and they matter more than the feature list:

• Over HTTP, fewer checks apply. Two checks (a secret leaking into output, and a leak into logs) look for a fake secret planted in the process the tool starts, so they are marked not applicable when your server is at a URL. They are listed, not counted.
• Through a proxy, the same two are lost. If you start your server via a stdio-to-HTTP proxy (mcp-remote for example), the process under test is the proxy. The tool recognises a short list of common proxies; a proxy not on the list, or a wrapper script around one, is not detected, and those two checks can then come out clean without meaning anything. A server inside a container or virtual machine is the same case, and that case was not tested.
• A clean result is not a certificate. It means these checks did not find a problem in this run, against this server, in this configuration.

Why a runtime check at all

When I ran the same battery against five widely used reference MCP servers, the one result that repeated everywhere was this: on every server that had a tool that changes data, the tool ran in a millisecond or two, with no error and no confirmation step. That is how the protocol is designed, and many clients ask the human for confirmation. But if your server is reached by a client that does not, nothing in the server stops it. That is a fact about behaviour, and it only shows up if you call the tool.

Try it, and tell me what is wrong

npx protocolprobe-selftest --i-own-this-server --stdio <your server command>

Source, issues and discussions: github.com/protocolprobe/protocolprobe-tools. If a check gives a wrong result on your server, that is the most useful bug report I can get.