如何構建可在 GitHub Actions runner 內運行的 AI 程式錯誤修復智能體

Disclosure: I'm building a product around this (bugzero). This article was drafted with AI assistance, then reviewed and edited by me.

Production errors are a good fit for an AI agent. There is a stack trace, there is a repository, and "done" has a clear definition: a pull request that a human reviews.

What bothered me was the usual shape of these tools. You give a SaaS read access to your repositories, and often your AI key as well, and hope for the best. So I started from one rule:

The service that coordinates the agent should never be able to read the code.

Everything below follows from that rule. I'll walk through the design, the code that enforces it, and the things that surprised me.

The shape of it

Two parts. A control plane knows about errors, configuration and run history. A runner, your own GitHub Actions runner, has the code and the AI key.

 Sentry / PostHog / webhook
          │  new error
          ▼
  control plane ── workflow_dispatch ──▶ your GitHub Actions runner
  (errors, config, run history)           · checks out your repo
          ▲                               · runs the agent (your AI key)
          │                               · opens a pull request
          └── status · PR link · summary · token counts ◀──┘

1. An error arrives from the error tracker.
2. The control plane picks the mapped repository and starts a workflow in it.
3. In your runner, the agent explores the checked-out code and writes a fix.
4. The fix goes to a new branch, and a pull request is opened.
5. The runner reports status, PR link, a summary and token counts back.

The control plane can only press a button

The control plane is a GitHub App with two permissions: Metadata: read and Actions: read & write. No Contents. It cannot clone your repository or read a file. All it can do is start a workflow with workflow_dispatch.

The workflow lives in your repo, so you can read exactly what runs:

on:
  workflow_dispatch:
    inputs:
      run_id:
        description: bugzero run id
        required: true
        type: string
      base_branch:
        description: Branch to fix
        required: true
        type: string
      token:
        description: Per-run callback token
        required: true
        type: string
      control_plane:
        description: URL to report back to
        required: false
        type: string

permissions:
  contents: write
  pull-requests: write

jobs:
  fix:
    runs-on: ubuntu-latest
    timeout-minutes: 25
    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ inputs.base_branch }}
          fetch-depth: 0
      - uses: runde-software/bugzero-action@v1
        with:
          run-id: ${{ inputs.run_id }}
          callback-token: ${{ inputs.token }}
          control-plane: ${{ inputs.control_plane }}
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

The AI key is a secret in your repository. The control plane never sees it.

A token that is good for exactly one run

The runner needs to fetch the error context and report the result, and GitHub gives it no identity toward my service. So the control plane mints a per-run token and passes it as a workflow input:

• It is an HMAC signature over the run id and an expiry, valid for 30 minutes.
• It is scoped to one run. It lets the runner read that run's error context and post that run's result, nothing else.
• It is masked in the job logs (::add-mask::).

The part I had to think about is that workflow inputs are shown on the run page. Anyone who can read your repository's Actions runs can see the token until it expires. That is why it is short-lived and narrowly scoped, rather than a long-lived credential I hope nobody looks at.

An agent with four tools and no shell

The agent has four tools:

There is no shell and no way to make network requests of its own. The reason is that the model's input is partly attacker-influenced: error messages, stack traces and code comments can contain text written by someone who is not you. An agent that can only read files and produce a diff has a small blast radius, and the diff still goes through code review.

Reads go through a path guard that rejects ../ escapes out of the checkout:

private resolve(rel: string): string {
  const root = path.resolve(this.opts.cwd);
  const abs = path.resolve(root, rel);
  if (abs !== root && !abs.startsWith(root + path.sep)) {
    throw new Error(`Path "${rel}" is outside the repository.`);
  }
  return abs;
}

Search is git grep -l -I -F, capped at 20 file names. Fixes are written in one place only: create_pull_request creates a new branch, commits as a bot user, pushes, and opens the PR. It never pushes to your base branch and never merges.

The cost of this design is real: the agent cannot run your tests. It can read a test file and mention it, but it cannot execute anything. I think that is the right default for now, and I'm not sure it is the right default forever. More on that at the end.

The loop has a budget

The loop stops at 30 tool calls or 20 minutes of wall-clock time, whichever comes first. The workflow's job timeout is 25 minutes, which leaves headroom to open the PR and report back.

The detail that mattered: every model call gets only the time that is left, not a fixed timeout. From the loop (trimmed):

const remaining = deadline - Date.now();
if (remaining <= 0) {
  stopReason = TIME_LIMIT_REASON;
  break;
}

// Every call gets only the time left, so one slow response can't outlive the
// job and leave the run unreported.
turn = await adapter.next(AbortSignal.timeout(remaining));

Without that, one slow response at minute 19 can push the job past its timeout, and the run is never reported as finished.

Gotchas

1. GitHub refuses PRs from Actions by default. Repositories ship with "Allow GitHub Actions to create and approve pull requests" turned off, and permissions: pull-requests: write in the workflow cannot override it. The push succeeds, then opening the PR returns a 403. I now treat that as its own case: the branch stays pushed, the run fails with a message that says exactly which setting to flip, and the proposed PR text is kept so the work isn't lost. If an organization locks the setting, you can pass a fine-grained token instead.

2. Keep volatile data out of the runner. Model prices change often. Early on, the action computed the cost of a run itself, so every price change meant a new release. Now the action only reports the token usage of each model request, and the control plane prices the run with its own catalog. Adding a model or changing a price needs no action release.

3. Newer OpenAI models do tool calls through the Responses API. The newest models I support don't call tools through Chat Completions, so the OpenAI adapter uses the Responses API. If you support several providers, keep each one behind a small adapter that returns the same normalized "turn" (text, tool calls, usage).

4. JavaScript actions run straight from the repository. There is no install step, so the bundled dist/index.js has to be committed. Users reference a moving major tag (@v1), and I tag each release as well so it can be pinned.

5. git add -A is a trap. The PR commit stages everything in the working tree. If someone adds a build step before the action, its output ends up in the pull request. The workflow keeps that job minimal, and the docs say why.

What leaves the runner

The claim "your code never leaves your infrastructure" would be wrong, so here is the precise version:

The control plane never receives your source code. The agent's summary is the model's own text, so it can mention file names or short fragments.

What it doesn't do

• It doesn't run your tests (see above).
• It doesn't guarantee a correct fix. The default instructions ask for a minimal diff, no new dependencies and no test edits, and when the agent isn't confident it should open a PR with only a defensive guard and an "Uncertain fix" section saying what the reviewer must check.
• It never merges. Everything arrives as a pull request.

Questions I would like your opinion on

1. Would you let an agent open PRs from production errors in your repositories? What guardrails would you require?
2. Should it be able to run tests before opening the PR? That means giving it a shell, which undoes the small blast radius. A sandboxed test step with no secrets might be the middle ground.
3. Is a path guard enough, or would you also resolve symlinks and run the job in a container without secrets?

The action's source is MIT-licensed: runde-software/bugzero-action. I'd like to hear where you think the design is wrong.