比較 Claude 與 ChatGPT 前，先用故障搜尋框測試編碼工具

A polished search box is a weak coding-model benchmark.

It can look right, return results and pass a happy-path demo while showing the wrong results to someone who types quickly.

Before comparing generated screenshots, give the tools a failure they have to explain. Here is one small, reproducible example. This article does not rank current model versions: I have not run a Claude-versus-ChatGPT comparison for it.

The bug hides in a reasonable-looking function

let visibleResults = [];

async function search(query, request) {
  const results = await request(query);
  visibleResults = results;
}

The user types re, then react. Both searches start. The response for react arrives first; the older response for re arrives last.

The input still says react. The result list now belongs to re.

Every request succeeded. The interface is still wrong.

Reproduce it without a network or a timer

const pending = new Map();
const request = query => new Promise(resolve => {
  pending.set(query, resolve);
});

const first = search('re', request);
const second = search('react', request);

pending.get('react')(['React result']);
await second;
pending.get('re')(['Older result']);
await first;

console.log(visibleResults); // ['Older result']

This deliberately controls completion order. A timer-based test might accidentally put the responses in the convenient order and miss the bug.

Require a rule before accepting a patch

For this example, the rule is: only the latest search is allowed to replace the visible results.

One small implementation uses a sequence number:

let latestSearch = 0;
let visibleResults = [];

async function search(query, request) {
  const sequence = ++latestSearch;
  const results = await request(query);
  if (sequence !== latestSearch) return;
  visibleResults = results;
}

Run the same controlled-order fixture with this implementation. The final result remains ['React result']. I checked both outcomes with executable assertions.

This is intentionally a minimal example. It omits rendering, loading state, error presentation, cancellation and component cleanup. Those concerns still need their own decisions. In particular, decide what to show if the latest search fails: a previous successful response is not automatically the right answer to the latest query.

MDN documents Promise completion and AbortController for aborting supported requests. Cancelling obsolete work can be useful, but the UI still needs a clear rule about which work may update it.

Compare the reasoning you can inspect

Give each coding tool the same initial function and fixture. Ask it to describe the failure before editing, make a bounded fix, and add a check that fails with the original implementation.

Keep the proposed patch, the explanation and the test result. Check whether the explanation actually matches the code. Then change one condition: make the latest request fail, or start a third search before the others finish.

This tiny exercise cannot decide which model is best for your codebase. It can expose a weak comparison: judging generated interfaces only by how convincingly they demonstrate the successful path.

I'm Daniil. I write about frontend failure modes and practical engineering decisions. Follow for more worked examples.

Which bug would you add to a coding-tool evaluation: stale responses, duplicate submits, or lost edits?