Next.js Webhook 與 Cron Job 如何安全應對失敗：簽名、重試及漏跑告警

A webhook endpoint is a public URL that does real work when someone posts to it. A cron job is a function that only does its job if a scheduler remembers to call it. Both fail quietly: a forged request looks like a normal one, a retried delivery looks like a new one, and a cron that stopped running produces no errors at all.

This is how we handle those failures in the small Next.js backend behind AI Craftbook, on Vercel. One inbound webhook forwards support email. One weekly cron job checks GitHub for changes to the projects we list. Code was written with Claude Code under our direction. Every output below comes from a test or a script we ran. Known gaps in our code are called out where they come up.

The failure modes

1. Verify the signature on the raw body

Our inbound email provider, Resend, signs webhooks with Svix. The Svix manual verification docs define the scheme: the signed content is ${svix_id}.${svix_timestamp}.${body}, the key is the base64 part of a whsec_... secret, the algorithm is HMAC-SHA256, and the svix-signature header holds a space-separated list of versioned signatures such as v1,<base64>. Here is our verifier, unchanged:

import { createHmac, timingSafeEqual } from "node:crypto";

const TOLERANCE_SECONDS = 5 * 60;

/** Svix signature check (Resend webhooks are signed by Svix). Secret is the "whsec_..." value. */
export function verifyWebhook(body: string, headers: Headers, secret: string, now = Date.now()): boolean {
  const id = headers.get("svix-id");
  const timestamp = headers.get("svix-timestamp");
  const signatures = headers.get("svix-signature");
  if (!id || !timestamp || !signatures || !secret.startsWith("whsec_")) return false;
  const ts = Number(timestamp);
  if (!Number.isFinite(ts) || Math.abs(now / 1000 - ts) > TOLERANCE_SECONDS) return false;
  const key = Buffer.from(secret.slice("whsec_".length), "base64");
  const want = Buffer.from(createHmac("sha256", key).update(`${id}.${timestamp}.${body}`).digest("base64"));
  return signatures.split(" ").some((entry) => {
    const [version, sig] = entry.split(",");
    const got = Buffer.from(sig || "");
    return version === "v1" && got.length === want.length && timingSafeEqual(got, want);
  });
}

A few details matter more than they look:

• Raw body. The route calls await request.text() and verifies that string before JSON.parse. Resend's docs warn that parsing and re-serialising the JSON can break the signature.
• Any listed signature may match. The header can carry several signatures, for example while a secret is being rotated. Our unit test signs with v1,bogus v1,<real> to prove a bad first entry does not cause a rejection.
• Length check before timingSafeEqual. Node's timingSafeEqual throws a RangeError when the buffers differ in length (we checked, output below). Comparing lengths first turns that into a clean false.
• Missing config fails closed. If the secret is not set, the route returns 503 and never processes the body.

We ran this exact function against a fabricated payload and a fake secret (FAKE_SIGNING_KEY_FOR_DOCS), importing it by path with tsx:

valid             true
tampered body     false
stale (10 min)    false
edge (4m59s old)  true
wrong secret      false
same delivery x2  true true

The last line comes up again in section 2.

2. The replay window, and what it does not stop

The timestamp is part of the signed string, so an attacker cannot refresh it without the secret. We reject anything more than five minutes away from our clock in either direction, which matches the window Svix's own libraries use according to their docs.

That narrows a replay to five minutes, but it does not prevent one. The same delivery x2 true true line shows it: the same signed request is accepted twice inside the window. A signature proves who sent the request. It does not prove the request is new. Making duplicates harmless is a separate job, covered in section 4.

3. Answer fast, and fail on purpose when you want a retry

Svix treats any 2xx as delivered and anything else (including redirects, or no answer within 30 seconds) as a failure. It then retries on a schedule documented in Svix's retry docs: immediately, then 5 seconds, 5 minutes, 30 minutes, 2 hours, 5 hours, 10 hours and 10 hours apart. Resend's own retry docs list the same schedule. Our route uses that on purpose. Trimmed, with environment lookups removed:

export const runtime = "nodejs";
export const maxDuration = 60;

export async function POST(request: Request) {
  // ... 503 if the signing secret is not configured
  const body = await request.text();
  if (!verifyWebhook(body, request.headers, secret)) return Response.json({ error: "bad signature" }, { status: 401 });

  const event = JSON.parse(body);
  if (!isForSupport(event)) return Response.json({ ok: true, skipped: true });
  const result = await forwardReceived(event.data.email_id, { /* ... */ });
  // A non-2xx makes Resend retry the webhook.
  return Response.json(result, { status: result.sent ? 200 : 502 });
}

Events we do not care about get a 200, so they are not retried. A failed forward gets a 502, so it is. Attachments are capped at 30 MB in total, which limits how much one email can make the handler download.

There is a catch here. Our handler does its work inline: it fetches the email, downloads the attachments and sends the copy before it answers. The function may run for up to 60 seconds, but Svix stops waiting after 30. If a forward takes longer than that, or succeeds but the response is lost, the provider records a failure and sends the event again. The email is then forwarded twice.

4. Idempotency: our gap, and the standard fix

We checked: our inbound route does not de-duplicate on svix-id. A retried delivery runs the forward again. For support email, the cost is a duplicate message in an inbox. It is not a security hole, because every delivery still has to pass the signature check. It is still a gap, and if the webhook moved money or changed records it would be a serious one.

The Svix docs say the message ID stays the same when a message is re-sent after a failure, so it works as an idempotency key. The usual pattern is to claim the ID in a table with a unique constraint before the side effect runs, and to release the claim if the side effect fails so the next retry can try again.

Not yet in our code; illustrative. We ran this sketch against in-memory SQLite (node:sqlite, Node 24). It has not been run on a production database:

import { DatabaseSync } from "node:sqlite";
const db = new DatabaseSync(":memory:");
db.exec("create table webhook_deliveries (id text primary key, received_at text not null)");
const claim = db.prepare("insert into webhook_deliveries (id, received_at) values (?, ?) on conflict (id) do nothing returning id");
const release = db.prepare("delete from webhook_deliveries where id = ?");

async function handle(msgId: string, send: () => Promise<void>): Promise<number> {
  if (!claim.get(msgId, new Date().toISOString())) return 200; // seen before: acknowledge, do nothing
  try {
    await send();
    return 200;
  } catch {
    release.run(msgId); // let the provider's retry try again
    return 502;
  }
}

Output from our demo, which counts sends:

first delivery   200 sent = 1
retry, same id   200 sent = 1
new id, fails    502 sent = 1
retry of msg 2   200 sent = 2

Postgres supports the same insert ... on conflict do nothing returning form (we have not run it there). This is "at most once per ID", not "exactly once". If the process dies after the send but before it answers, the claim is kept and the retry is skipped. For email forwarding, that is the failure we would choose.

There is a second trade-off, and it hits exactly the slow-handler case from section 3. If a retry arrives while the first attempt is still running, the claim already exists, so the retry gets a 200. If the first attempt then fails and releases its claim, the provider has already recorded success and will not try again, so the message is lost. We checked this with the same sketch: the overlapping retry got 200 and the first attempt got 502. One fix is to store a status with each claim and answer a non-2xx while a claim is still marked in progress, so the provider keeps retrying.

5. Authenticate the cron endpoint

A cron route is a public URL too. Vercel calls it with an HTTP GET to the production deployment, on a schedule in UTC (Vercel cron docs). Our vercel.json has a single cron entry (shown trimmed, with the path shortened):

{
  "crons": [{ "path": "/api/cron/...", "schedule": "0 13 * * 1" }]
}

That is Mondays at 13:00 UTC. Vercel's docs say that if the project has an environment variable named CRON_SECRET, Vercel sends it as Authorization: Bearer <value>. They recommend a random value of at least 16 characters. Their sample compares the header with !==. Ours is stricter. This is our check, with the environment lookup moved out so the secret is a parameter:

import { timingSafeEqual } from "node:crypto";

function authorized(header: string | null, secret: string | undefined): boolean {
  if (!secret || secret.length < 32) return false;
  const got = Buffer.from(header || "");
  const want = Buffer.from(`Bearer ${secret}`);
  return got.length === want.length && timingSafeEqual(got, want);
}

We require a shared secret of at least 32 characters. If the secret is missing or too short, the endpoint refuses every request instead of falling back to open. Output with a fake 36-character token:

correct header    true
no header         false
wrong, same len   false
wrong length      false
secret unset      false
secret too short  false
timingSafeEqual on unequal lengths throws: RangeError

Two more points from the same Vercel page shape everything below. Vercel does not retry a failed cron invocation. Delivery is best effort: a run can be missed, and occasionally the same run is invoked twice. So a cron job has to cope with missed runs and duplicate runs.

6. Retry outbound calls, but only the retryable ones

The weekly job reads repository metadata from the GitHub REST API. This is the core of our client:

async api<T>(pathname: string, etag?: string): Promise<ApiResult<T>> {
  for (let attempt = 0; ; attempt++) {
    this.budget.take();
    await sleep(this.opts.delayMs);
    const res = await fetch(`https://api.github.com${pathname}`, { headers: this.headers(etag), cache: "no-store" });
    if (res.status === 304) return { status: 304, etag, notModified: true };
    if ((res.status === 403 || res.status === 429 || res.status >= 500) && attempt < this.opts.maxRetries) {
      await sleep(2000 * (attempt + 1));
      continue;
    }
    const out: ApiResult<T> = { status: res.status, etag: res.headers.get("etag") || undefined };
    if (res.ok) out.data = (await res.json()) as T;
    return out;
  }
}

We retry 403, 429 and 5xx, because GitHub documents 403 or 429 for rate limits. We do not retry other 4xx answers: a 404 on the repository is treated as a deletion and reported as a change, not retried. (GitHub's best-practices page notes that a 404 can also come from missing access to a private resource, so check credentials before trusting it.) We ran the real class with fetch replaced by a local fake (no network, retries set to 2, delay 0):

1) 503 then 200
  +0.0s fake GitHub -> 503
  +2.0s fake GitHub -> 200
  result: {"status":200,...,"data":{"ok":true}} | budget used: 2
2) 429 three times (retries exhausted)
  +0.0s fake GitHub -> 429
  +2.0s fake GitHub -> 429
  +6.0s fake GitHub -> 429
  result: {"status":429,...} | budget used: 3
3) 404 (not retried)
  +0.0s fake GitHub -> 404
  result: {"status":404,...} | budget used: 1

The output also shows where our client falls short of GitHub's guidance. The backoff is linear (2 s, then 4 s), not exponential. We do not read retry-after or x-ratelimit-reset, which GitHub's best-practices page says to wait for. It says to wait at least a minute when neither header is present. We also retry every 403, including permission errors that will never succeed. For a once-a-week job with a few retries, the cost is small. For a busier job it would matter.

7. Conditional requests: let unchanged data cost nothing

We store the etag from each response and send it back as If-None-Match next week. GitHub answers 304 Not Modified with no body if nothing changed. According to GitHub, a 304 does not count against the primary rate limit when the request was authorized with an Authorization header. Scenario 4 from the same demo:

4) conditional request answered 304
  +0.0s fake GitHub -> 304 (If-None-Match sent)
  result: {"status":304,"etag":"W/\"fake-etag-1\"","notModified":true} | budget used: 1

The caller has to treat notModified as "same as last time", not as an error. Our source checker does that. The docs-page checker in the same job does the same with If-None-Match and If-Modified-Since, and a unit test asserts that the second request sends the stored ETag and reports no change.

8. Put a hard cap on every run

Each request goes through a counter that throws when a per-run cap is exceeded. The cap is set in a config file, not in code. Retries draw from the same budget, so a retry storm cannot exceed it. Scenario 5, with the cap set to 2:

5) budget of 2 requests, 500 every time
  +0.0s fake GitHub -> 500
  +2.0s fake GitHub -> 500
  threw: github: request cap 2 reached. Raise it in pipeline.config.json if this is expected.

In the GitHub source checker, each source has its own try/catch, so hitting the cap shows up as an error for each remaining source. Those errors go into the run's alert email, so the run never reports success when it skipped work.

9. Detect the cron that silently stopped

Because Vercel does not retry and does not promise delivery, the job writes a row at the start of every run. It then reads the start time of the previous run:

/** A scheduled run is weekly; a gap longer than this means at least one run did not happen. */
export const MISSED_RUN_DAYS = 8;

if (gapDays !== null && gapDays > MISSED_RUN_DAYS) lines.push(`The previous monitor run was ${Math.floor(gapDays)} days ago. At least one weekly run did not happen.`);

Eight days gives a weekly job a day of slack before it counts as missed. The same alert lists errors from the run, so a failed GitHub call (a 403, say) is reported instead of passing as "no changes". The run also acts on what it finds. If a project we redistribute changes in a security-relevant way (for example, its repository is deleted or files change in a folder whose text we show), the job withdraws that download from the site straight away, without waiting for an editor.

The limit is that the gap check runs inside the job, so a missed run is only reported when the next run happens. If the schedule stops completely (crons disabled, the entry removed in a deploy), nothing ever reports it. The fix is an external check, often called a dead man's switch: something outside the job that raises an alarm when a heartbeat stops arriving. We do not have one. [TIM: confirm there is no external heartbeat on the Vercel cron, or describe it if there is.]

Testing these paths

Most of the behaviour above has unit tests that need no network or secrets. Fakes stand in for fetch and for GitHub:

• inbound.test.ts: accepts a valid signature (with a bogus first entry), rejects a tampered body, a 10-minute-old timestamp and missing headers. Forwarding sets Reply-To to the sender and does nothing without config.
• ops.test.ts: a deleted repository is a security change. A 403 is reported as an error, not a silent pass. A shared repository is fetched once per run. Alerts fire for security changes, errors and a gap of MISSED_RUN_DAYS + 1, but not for a new release on its own.
• monitor-docs.test.ts: conditional headers are sent and a 304 counts as unchanged.
• mcp-monitor.test.ts: package-level changes (new maintainers, removed packages, advisories) are sorted by risk.

On the day of writing, npx vitest run on these four files passed 35 of 35 tests. One gap: the retry and budget logic in the GitHub client has no unit test of its own. The ops tests use a fake API object that skips it. The demo in section 6 is currently its only check.

Limits

• No webhook de-duplication yet (section 4). Duplicate forwards are possible. The simple claim-then-act sketch also has an overlap case to handle before it ships.
• The webhook does its work inline. A slow forward can exceed the provider's 30-second wait and trigger a retry.
• Retry backoff is linear and ignores GitHub's rate-limit headers. Permission 403s are retried pointlessly.
• No lock against two runs of the cron at once, which Vercel's docs say can happen. Our run writes state with upserts, but a duplicate run would probably send a second alert email. We have not tested that.
• The missed-run alarm fires only on the next run. There is no external heartbeat.
• We have not run any of this end to end against production failures. The results here come from unit tests and the local scripts shown.

This article was drafted with AI assistance (Claude Code) from our project's code and test records, and reviewed and edited by Tim S before publishing.