串流聊天重試按鈕重送同一提示詞：冪等性除錯指南

The bug report you will eventually get is short: “Retry sometimes gives me two answers.” The transcript looks normal until you notice two assistant bubbles with almost identical text and a duplicate “Generating response” announcement in the screen-reader log. Nothing threw an exception, so the console is useless. Where do you even start?

I reduce this class of failure to two requests and one question: is this retry the same user intent, or a new one? A retry after a dropped connection is the same intent. A regenerate after a successful answer is a new intent. When the client treats both as “send again,” you get duplicate work, duplicate announcements, and sometimes duplicate billing.

The failure is two different requests wearing one button

The state table I want on the wall before touching code:

The table makes the root cause less mysterious. The server cannot deduplicate two requests if the client gives them different identities. The button also cannot prevent a second submit if it is disabled only in CSS. Those are separate defects, and fixing one leaves the other alive.

Reproduce it with one idempotency key

The smallest server I use for this review is an in-memory ledger with an SSE response. It is not production storage, but it makes the contract visible. Node 20 or newer is enough; there are no dependencies.

// server.mjs
import { createServer } from 'node:http';
import { createHash } from 'node:crypto';

const ledger = new Map();
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

function sendEvent(res, event, data) {
  res.write(`event: ${event}\ndata: ${JSON.stringify(data)}\n\n`);
}

function tokenize(prompt) {
  return ['A ', 'cached ', 'answer ', 'for: ', prompt];
}

const server = createServer(async (req, res) => {
  if (req.method !== 'POST' || req.url !== '/api/chat') {
    res.writeHead(404).end('Not found');
    return;
  }

  let body = '';
  for await (const chunk of req) body += chunk;

  let payload;
  try {
    payload = JSON.parse(body || '{}');
  } catch {
    res.writeHead(400, { 'content-type': 'application/json' });
    res.end(JSON.stringify({ error: 'invalid json' }));
    return;
  }

  const prompt = typeof payload.prompt === 'string' ? payload.prompt : '';
  const key = req.headers['idempotency-key'];
  if (!prompt || typeof key !== 'string' || key.length < 8) {
    res.writeHead(400, { 'content-type': 'application/json' });
    res.end(JSON.stringify({ error: 'missing prompt or idempotency key' }));
    return;
  }

  const fingerprint = createHash('sha256').update(prompt).digest('hex');
  const prior = ledger.get(key);

  if (prior && prior.fingerprint !== fingerprint) {
    res.writeHead(422, { 'content-type': 'application/json' });
    res.end(JSON.stringify({ error: 'key reused for a different prompt' }));
    return;
  }

  if (prior?.state === 'running') {
    res.writeHead(409, { 'content-type': 'application/json' });
    res.end(JSON.stringify({ error: 'request already running' }));
    return;
  }

  if (prior?.state === 'done') {
    res.writeHead(200, {
      'content-type': 'text/event-stream',
      'cache-control': 'no-cache',
      'idempotent-replay': 'true',
    });
    sendEvent(res, 'token', prior.text);
    sendEvent(res, 'done', {});
    res.end();
    return;
  }

  ledger.set(key, { state: 'running', text: '', fingerprint });
  res.writeHead(200, {
    'content-type': 'text/event-stream',
    'cache-control': 'no-cache',
    connection: 'keep-alive',
  });

  let text = '';
  for (const token of tokenize(prompt)) {
    await sleep(250);
    text += token;
    sendEvent(res, 'token', token);
  }

  ledger.set(key, { state: 'done', text, fingerprint });
  sendEvent(res, 'done', {});
  res.end();
});

server.listen(8787, () => console.log('http://localhost:8787'));

Run it, then fire two requests with the same key at the same time:

node server.mjs

URL=http://localhost:8787/api/chat
KEY=6f1c2f6e-7a1b-4a4e-9d1a-111111111111
BODY='{"prompt":"accessibility"}'

for i in 1 2; do
  curl -sS -D headers-$i.txt -o body-$i.txt -X POST $URL -H 'content-type: application/json' -H "idempotency-key: $KEY" -d "$BODY" &
done
wait
grep -iE 'HTTP/|idempotent-replay' headers-*.txt

Expected result: one 200, one 409. Run the same curl again after the stream finishes, and you should see idempotent-replay: true. The second request does not call the generator again. That is the whole trick, and it has nothing to do with a particular model vendor.

Now break it on purpose. In the client, generate a new key inside the retry handler and remove the in-flight guard. The server sees two keys, so it treats them as two user intents. Two answers are then correct behavior for the server and wrong behavior for the product.

The client fix: reuse the key, guard the click

The key belongs to the message attempt, not to the fetch call. Create it when the user submits a new prompt. Reuse it for retries caused by a network error, a double Enter, or a refresh that resumes the same attempt. Create a new key only when the user chooses “Regenerate” after a completed answer.

The identity diagram is small enough to keep beside the code:

new prompt     ──► key A ──► POST ──► 200 stream ──► done
network retry  ──► key A ──► POST ──► 200 replay (same answer)
regenerate     ──► key B ──► POST ──► 200 stream (new answer)

Here is the smallest client state I use. The HTML keeps the retry control in the document and uses aria-disabled instead of disabled, so focus does not vanish mid-interaction.

<form id='chat'>
  <label for='prompt'>Prompt</label>
  <textarea id='prompt' required></textarea>
  <button id='send' type='submit'>Send</button>
</form>
<p id='status' role='status'></p>
<section id='answer' hidden aria-labelledby='answer-heading'>
  <h2 id='answer-heading' tabindex='-1'>Answer</h2>
  <p id='answer-text'></p>
  <button id='retry' type='button' aria-describedby='status' hidden>Retry</button>
</section>

<script>
  const form = document.querySelector('#chat');
  const promptEl = document.querySelector('#prompt');
  const send = document.querySelector('#send');
  const retry = document.querySelector('#retry');
  const statusEl = document.querySelector('#status');
  const answer = document.querySelector('#answer');
  const answerText = document.querySelector('#answer-text');
  const answerHeading = document.querySelector('#answer-heading');

  let requestKey = null;
  let inFlight = false;

  function setBusy(isBusy) {
    inFlight = isBusy;
    send.toggleAttribute('aria-disabled', isBusy);
    retry.toggleAttribute('aria-disabled', isBusy);
  }

  async function post(prompt) {
    if (inFlight || !requestKey) return;
    setBusy(true);
    statusEl.textContent = retry.hidden ? 'Sending.' : 'Retrying the same request.';

    try {
      const response = await fetch('/api/chat', {
        method: 'POST',
        headers: {
          'content-type': 'application/json',
          'idempotency-key': requestKey,
        },
        body: JSON.stringify({ prompt }),
      });

      if (response.status === 409) {
        statusEl.textContent = 'That request is still running. Wait for the current answer.';
        return;
      }
      if (!response.ok) throw new Error(`HTTP ${response.status}`);

      answer.hidden = false;
      answerText.textContent = '';
      const reader = response.body.getReader();
      const decoder = new TextDecoder();
      let buffer = '';

      while (true) {
        const { value, done } = await reader.read();
        if (done) break;
        buffer += decoder.decode(value, { stream: true });
        const frames = buffer.split('\n\n');
        buffer = frames.pop() ?? '';

        for (const frame of frames) {
          if (!frame.startsWith('event: token')) continue;
          const line = frame.split('\n').find((part) => part.startsWith('data: '));
          if (line) answerText.textContent += JSON.parse(line.slice(6));
        }
      }

      statusEl.textContent = 'Answer complete.';
      if (document.activeElement === retry) answerHeading.focus();
      retry.hidden = true;
    } catch (error) {
      statusEl.textContent = `Answer failed: ${error.message}. You can retry this same request.`;
      retry.hidden = false;
      retry.focus();
    } finally {
      setBusy(false);
    }
  }

  form.addEventListener('submit', (event) => {
    event.preventDefault();
    if (inFlight) return;
    requestKey = crypto.randomUUID();
    post(promptEl.value);
  });

  retry.addEventListener('click', () => {
    if (inFlight) return;
    post(promptEl.value);
  });
</script>

The guard runs in two places because a double click can beat a visual update. The inFlight flag catches the second click before aria-disabled is painted. The aria-disabled attribute tells assistive technology that the action is temporarily unavailable, while the element remains focusable. Test that behavior with your actual screen reader; support is not uniform.

The focus move after a successful retry is deliberate. If the retry button disappears while it owns focus, the next Tab starts from the top of the document. Moving focus to the answer heading gives the user a stable landmark. If your action bar stays visible, keep focus there and change the label to “Regenerate” instead, but assign a new key only when the user actually asks for a new generation.

What the screen reader and keyboard need

The visible error and the announced error should be the same sentence. In the snippet, aria-describedby='status' points at the polite status region, so a retry button does not announce a stale message from an earlier failure. When the retry begins, the status changes once to “Retrying the same request.” Do not stream every token into that live region; announce state transitions, not the answer itself.

A keyboard-only pass should cover the exact sequence that breaks the UI:

1. Tab to Retry and press Enter twice within 300 ms.
2. Confirm the Network panel shows one POST, not two.
3. Confirm one assistant answer exists after completion.
4. Force a 500 before the first token, then retry with Enter.
5. Confirm focus remains on Retry while busy, then lands on the answer heading after success.
6. Repeat with a screen reader and record the announcement order.

If the second Enter creates a second request, the bug is not “the model is slow.” The client created a second identity. If the screen reader announces completion twice, the live region is probably receiving both a token event and a done event. Those are different fixes, and the state table keeps them separate.

A QA matrix you can repeat

Record the browser, OS, and assistive technology versions beside each row. A result without those versions is hard to reproduce, and “works on my machine” is not a QA result.

Where MonkeyCode fits, and where this design stops

The reproduction above uses a local generator so it runs without an account. When I want to exercise the same ledger with a real model, one operator-supplied option is MonkeyCode: the operator describes the project as open source, with free model access and a free server option, including 10 million free tokens. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I am not naming models, quoting rate limits, or promising uptime, because I have not verified those details and they are not required to debug the retry contract.

Is that a good fit for this test? It can be, if you need a place to host the proxy and a model endpoint for a prototype. It is not a substitute for durable idempotency. The ledger above lives in memory, so a deploy or crash forgets every key. For production, put a unique constraint on (account_id, idempotency_key) in a database and store the terminal response with a short expiry. If prompts can contain personal data, store only the status and a hash, or return 409 instead of replaying the content.

Who should not use this approach? Teams that need exactly-once behavior across regions, teams that cannot persist a key, and teams whose provider cannot replay or cache a completed answer. In those cases, make Retry a new user-visible action with a new key, and say so in the label. Do not pretend a network retry is free when it can create a second answer.

If you try the workflow on a free deployment, bring back the browser, OS, assistive technology, and the exact transition that failed. A screenshot of two answers is a symptom. The request keys in the Network panel are the evidence.