語音 AI 數字欄位應以驗收測試取代籠統準確率口號

Voice AI Numbers Need an Acceptance Test, Not an Accuracy Slogan

The short answer

Do not accept a voice agent's handling of phone numbers, account references, dates, claim identifiers, or payment amounts because a vendor quotes one overall speech-recognition accuracy figure. Define the field, its allowed normalization, its consequences, and the evidence needed before the value may be stored or acted upon. For a reversible, low-impact field, a constrained parser plus a lightweight confirmation may be enough. For a consequential identifier or amount, capture the value in a typed slot, preserve the raw hypothesis, validate it deterministically, read back the appropriate representation, obtain an explicit correction opportunity, and prevent a downstream action until the confirmed value is the value actually sent.

That is not a claim that reading every digit aloud is always safer. It is a proposal for an acceptance contract: measure exact-field integrity on the actual channel and task, and make the confirmation burden proportional to the cost of a wrong value. A phone number may need country-aware normalization; an account number may need leading zeroes preserved; a spoken amount may need currency and decimal interpretation. These are different data types, not one generic “number” slot.

What a reader should decide

The decision is whether a voice workflow may submit, save, route, or use a number without human intervention, and what must happen first. Before a pilot, the owner should be able to answer four questions in writing:

1. What exactly is the field: a telephone contact, a fixed-width account code, a date, a currency amount, or an unconstrained numeric phrase?
2. Which transformations are permitted, and which must be rejected rather than guessed?
3. What is the consequence of a plausible but wrong value, including privacy exposure and recovery cost?
4. What observable evidence proves that the user confirmed the same canonical value that the downstream system received?

If any answer is unknown, the safe state is “do not commit yet.” That is not a general prohibition on voice automation. It is a separation between collecting a candidate and authorizing a side effect.

Why a single accuracy number is the wrong acceptance criterion

The source page, Speech Recognition Technology 2026: Advancements, presents a broad ASR pipeline: audio capture, preprocessing, acoustic and language modeling, then punctuation and formatting. It also states a 95–98% real-world accuracy range for leading systems. The page does not define that range beside the claim: it does not identify the transcript corpus, language mix, channel, metric, confidence interval, annotation policy, or whether the unit is words, characters, entities, or complete tasks. Treat the percentage as an unverified overview claim, not as an acceptance threshold for your workflow.

The distinction matters mathematically. Word error rate is generally calculated from substitutions, deletions, and insertions against a reference transcript, divided by the number of reference words. A field-level exact-match measure asks a different question: did this particular account code, phone number, or amount equal the expected canonical value? A transcript can have low aggregate word error while one critical digit is wrong. Conversely, punctuation or filler-word differences can increase a transcript metric without changing a low-risk field. Neither metric replaces the other.

There is a simple sequence effect too. If each digit were independently correct with probability p, a sequence of n digits would be exactly correct with probability p^n. At a hypothetical 99% per-digit accuracy, an 8-digit string would have an exact-match probability of about 92.3% under that simplifying independence assumption. This is an illustration, not a measured voice-system result: real errors are correlated, digits are not always recognized independently, and the starting p is not asserted for any product. It shows why token averages can obscure the all-or-nothing property of a reference number.

The measurement unit must therefore match the business decision. If the action requires the entire identifier to be correct, report the exact-sequence rate. If a caller can correct the value before it is committed, also report recovery success and correction burden. If the system normalizes a phone number, measure both the raw transcription and the normalized result; do not let formatting success hide digit loss.

Separate transcription, interpretation, and commit

A reliable design does not ask one generative component to “hear correctly” and then immediately execute a tool. It creates observable boundaries:

Each boundary should be inspectable after a test. If the stored phone number is wrong, an operator should be able to determine whether the recognizer misheard, the formatter changed the value, a parser removed a leading zero, the agent read a different string, or an integration sent a stale value. A transcript alone cannot answer those questions if the system discarded intermediate state.

Typed fields beat a generic “number” prompt

A telephone number is often spoken as digits, pairs, or familiar groups. It may need a country code and extension. An account reference may look similar but be a fixed string where leading zeroes are meaningful and grouping has no semantic significance. An amount may require a currency and decimal separator. A date may have locale ambiguity: “04/05” can mean different days and months. Store each as a type with explicit normalization rules.

Do not call a string “valid” simply because it contains ten digits. A regex can enforce shape, not ownership, intent, country, or semantic correctness. A checksum can detect some transcription errors, not prove the user gave the right identifier. An API lookup can confirm that a reference exists, not that it belongs to the caller. The validation layer narrows possible errors; confirmation and authorization still have separate jobs.

Raw, normalized, and confirmed are different states

Keep three values separate where the data policy permits it:

• raw: the recognition output as produced, with provenance and access controls;
• candidate: the deterministic, typed normalization proposed by the application;
• confirmed: the value the user accepted or explicitly corrected, bound to the pending action.

Do not overwrite raw with a prettified version. Do not treat candidate as confirmed. Do not let a later language-model paraphrase replace the value stored by a deterministic parser. If privacy or retention rules mean the raw audio/transcript cannot be kept, document that limit and retain the smallest lawful audit record: field type, normalization version, confirmation event, payload digest, outcome, and deletion schedule. Data minimization is not an excuse to make an action unauditable; auditability is not an excuse to retain call audio forever.

What current guidance and user reports do—and do not—establish

The official LiveKit tasks documentation describes focused reusable task units that return typed results and discusses task testing. This supports a useful architecture principle: make a collection step return a typed result, then validate it before another task or tool acts. It is framework documentation, not evidence that a particular ASR model recognizes digits accurately. A typed result can make errors easier to contain; it cannot make recognition correct by itself.

An AssemblyAI benchmark page advertises voice and speech benchmarks, including model and task comparisons. It is a vendor-published benchmark resource. Use it to discover that task definitions and datasets matter, not as a neutral universal ranking. Before using any vendor figure, read its protocol: data source, language, audio conditions, benchmark version, reference annotation, selection criteria, and the exact score being compared. A named-entity benchmark is closer to field capture than general transcription, but even that may not match your callers, identifier formats, telephony codec, consent model, or downstream consequences.

Real user discussions reveal questions that aggregate product descriptions tend to omit. In a May 2026 Genesys developer-community thread, a user described a phone-number slot that reformatted a ten-digit example and asked the caller to confirm only the last four digits, while another numeric account slot behaved differently. A staff response first recommended a regular expression; the original poster replied that a regex already enforced the length, and later replies discussed phone-number-specific semantics. This is one product-specific report, not a prevalence study. It illustrates a design gap: correct length and a partial read-back do not establish that the entire intended field was confirmed.

The research record also warns against treating entity capture as a formatting afterthought. A 2022 telephone-conversation NER paper describes ASR noise and annotation errors in its training data, then evaluates a human-in-the-loop re-annotation approach; its reported improvement concerns one entity type in that research setup, not voice-agent digits or our proposed suite. An older IBM-affiliated Interspeech paper directly examines symbol sequences such as phone and customer-identification numbers in telephone conversations, including fragments and repeated sequences. It reports an F-measure for an eight-digit ID search task at a stated WER, but that task-specific result from 2017 is not a modern product comparison. These studies support the need for task- and entity-specific evaluation; their populations, systems, and metrics must not be transplanted to a current deployment. See Laskar et al. in the ACL Anthology and Suzuki et al. at IBM Research.

An October 2026 3CX forum discussion raises the inverse formatting issue: a user reports a phone number and other numeric sequences rendered as spoken words in a transcript, and asks whether a provider’s inverse-text-normalization option is exposed in a specific 3CX version. The post names a particular version/model and is a user report, not a controlled comparison. It demonstrates that representation itself can be part of the workflow: text that is readable as words may be hard to copy, while formatting words into digits can introduce an unverified transformation.

The two discussions appear contradictory only if we assume one universal desired output. They describe different field semantics and interface goals: one system may normalize a phone number and mask part of its confirmation, another may preserve word-form digits in a transcript. The proper resolution is not “always format” or “never format.” It is to declare the canonical representation per field and prove that any transformation preserves the intended value. The sources do not show how often either issue occurs, nor whether the same models or conditions were tested.

The Speechmatics LinkedIn video post is a vendor demonstration described as showing enhanced English handling for alphanumeric voice-agent work. A product demo can help a buyer identify interface questions—what value is shown, when correction is offered, and where a task stops—but it is not a benchmark. The post’s accompanying description does not establish sample size, ground truth, error rate, test selection, or a blinded comparison. Treat it as a demonstration of a product claim, not measured proof.

For deployments that fall within its scope, the European Commission’s AI Act overview describes a risk-based legal framework and identifies traceability, human oversight, robustness, cybersecurity, and accuracy among obligations for high-risk systems. This article does not classify a voice-number workflow under the Act; that depends on the actual intended use and other rules. The official overview is a reminder to perform a separate scope and compliance review, not evidence that this particular numeric field has a prescribed read-back design. On evaluation itself, Stanford HAI’s AI Index says its 2026 report tracks and visualizes AI data and flags declining transparency as a reason independent measurement matters. It is broad AI context, not voice recognition research. A broad EY AI insights page similarly covers organizational AI strategy and trust but contains no numeric-capture benchmark. It is consulting context, not technical validation. These qualifications matter: law and consulting frameworks can shape governance, while only a field-matched test can answer whether a given stack preserves the value.

An original acceptance asset: a 15-case exact-field suite

The following is a proposed, reusable test plan, not an experiment already run. It contains fifteen cases selected to expose distinct failure classes. The count “15” is the size of the proposed fixture set, not a result, sample estimate, or statistical claim. Before using it, replace the example values with synthetic fixtures that match your permitted formats. Never use real customer identifiers in a public demo or test log.

For each case, capture the expected value, audio/locale conditions, transcript, parsed candidate, normalized value, confirmation event, final payload, and reason for pass/fail. A case passes only when the final payload equals the test fixture’s expected canonical value and the interaction obeys the privacy and consent policy. If an action is deliberately not taken because the input remains ambiguous, that is a safe non-commit result, not a recognition success. Report both categories separately.

The proposed suite deliberately includes user-correction and refusal behavior. A benchmark limited to clean, uninterrupted utterances misses the cases where actual workflows recover or fail. A successful product acceptance test should also include diverse but lawfully sourced accents, microphones, codecs, background noise, speech rates, and language variants relevant to the service. Those additions are local extensions, not included in the fifteen-case count above.

Score the field, not the demo

Keep metrics separate so a good average does not mask an unsafe tail:

• Exact candidate rate: candidates exactly equal the reference value before confirmation.
• Exact committed-field rate: committed payloads exactly equal the confirmed reference value.
• Silent mutation count: changes to length, leading zero, digit, sign, decimal, country code, or extension without explicit user correction.
• Safe non-commit rate: ambiguous or invalid fields correctly stop before side effect.
• Recovery rate: cases initially wrong or uncertain that reach the correct confirmed value without a forbidden commit.
• Correction burden: number of clarification turns and user actions required per successful field.
• Privacy-policy failures: unmasked read-backs, retention beyond the configured limit, or disclosure before required identity checks.

Do not collapse these into one weighted score before setting hard gates. A wrong account code that is detected and safely rejected is different from a wrong code sent to a payment tool. A conservative system may have a lower “first-pass exact candidate rate” yet a higher exact committed-field rate because it asks for clarification. That can be desirable if the extra burden remains acceptable. Conversely, confirmation prompts that callers routinely misunderstand are not an adequate control just because a click or “yes” was recorded.

A deterministic reference function

The snippet below is an intentionally narrow example for an English-language fixed-length digit string. It accepts only a sequence containing numeric digits and spaces/hyphens or spoken single-digit English words. It preserves leading zeroes by returning a string, not a number. It does not guess “double four,” parse amounts, infer a country code, or turn natural-language numbers into identifiers. Ambiguity returns an error for clarification rather than a repaired value.

const WORD_TO_DIGIT = Object.freeze({
  zero: '0', oh: '0',
  one: '1', two: '2', three: '3', four: '4', five: '5',
  six: '6', seven: '7', eight: '8', nine: '9',
});

function parseDigitSequence(input, expectedLength) {
  const tokens = String(input).toLowerCase().trim().split(/[\s-]+/).filter(Boolean);
  if (!tokens.length) return { ok: false, reason: 'empty' };

  let digits = '';
  for (const token of tokens) {
    if (/^\d+$/.test(token)) {
      digits += token;
      continue;
    }
    if (Object.hasOwn(WORD_TO_DIGIT, token)) {
      digits += WORD_TO_DIGIT[token];
      continue;
    }
    return { ok: false, reason: 'ambiguous-token', token };
  }

  if (!Number.isInteger(expectedLength) || expectedLength < 1) {
    return { ok: false, reason: 'invalid-schema' };
  }
  if (digits.length !== expectedLength) {
    return { ok: false, reason: 'length-mismatch', actualLength: digits.length };
  }
  return { ok: true, digits };
}

function confirmationMatches(candidate, confirmed) {
  return candidate.ok && confirmed.ok && candidate.digits === confirmed.digits;
}

In production, schema validation should be outside the model prompt, the parser version should be logged, and the confirmation event should bind to a particular candidate digest. This example does not establish the caller’s identity, detect fraud, secure a phone line, or implement language-specific number grammar. It is a control sketch, not a drop-in library. Most importantly, a matching read-back is not equivalent to a valid authorization: the user may not hear, understand, or be entitled to approve the next action.

Confirmation should reduce risk without creating new risk

There are several reasonable confirmation styles, and none is universally best:

The table is a decision aid, not a study result. Choose based on the sensitivity of the data, channel privacy, legal obligations, caller capability, and reversibility of the action. Reading a full bank or health identifier aloud into a shared space may increase exposure. A masked confirmation can reduce disclosure but may not let a caller catch an error in a hidden digit. A keypad can improve exactness for some users while excluding others. A human handoff is not safe unless the agent transfers context correctly and clearly says whether the action was or was not committed.

The control objective is not “make the user say yes.” It is: present the relevant value in a usable form, make correction possible, bind acceptance to the current candidate and intended action, then revalidate the exact outgoing payload immediately before the tool call. If the value changes after confirmation, invalidate the approval and ask again.

Failure modes that pass superficial reviews

A transcript looks right, but the slot is wrong

Some systems expose a transcript that differs from the value passed to a tool. Log both, with versions and transformation boundaries. Test the integration path, not only the speech model’s transcript display.

The slot is formatted, then mistaken for verified

Formatting a phone number with separators can improve legibility. It does not prove that every digit survived. Compare canonical digit strings separately and retain extension/country fields distinctly.

Regex validates length, not intent

A ten-digit pattern can reject a nine-digit value, yet still accept the wrong ten digits. Treat shape validation as one condition only. Keep identity proof, existence checks, caller intent, and authorization separate.

Confirmation repeats only a suffix

Partial confirmation may be a privacy-friendly user-interface choice, but it cannot establish full-string fidelity unless a different trusted mechanism checks the hidden portion. Do not advertise it as full confirmation.

A correction arrives during read-back

The system may continue speaking stale digits after the caller interrupts. Invalidate the old confirmation state when a correction begins, stop any pending side effect, and restart read-back with the updated candidate. Test barge-in, delayed tool responses, and the race between user correction and commit.

The retry policy hides exhaustion

Repeated “I did not get that” prompts can frustrate users and create accidental acceptance. Set a bounded retry policy per field. When exhausted, explain what remains incomplete and provide an alternate channel. Never fabricate a partial value to keep the conversation moving.

A number is converted to an integer

Numeric identifiers are strings. Converting 001234 to a number yields 1234; the lost zero cannot be recovered reliably. Store canonical digit sequences as strings, define length, and test leading zeroes.

Locale and number grammar are mixed

“Double,” “oh,” decimal separators, and grouped numbers vary by language, region, and context. A parser for English digit spelling is not a French, German, or Swiss number parser. Select locale explicitly, ask when the grammar is ambiguous, and version the rules. Do not expand the accepted grammar invisibly after a provider update.

A checksum becomes an authorization shortcut

A checksum catches some typing or transcription mistakes; it does not prove that a caller owns the reference. A successful database lookup can reveal whether an account exists. Apply identity and anti-enumeration controls before disclosing any associated data.

“The model was 99% accurate” has no denominator

Ask what was counted: words, utterances, entities, or complete transactions? Was the test telephony audio, which languages and accents, what annotation policy, how many speakers, and what happened to failures? If these are unavailable, the number is not usable to predict your field-level risk.

Synthetic fixtures hide real channel defects

Clean studio audio can test parser logic and cannot stand in for compressed phone audio, crosstalk, speakerphone echo, interruptions, distant microphones, and real speech rates. Use synthetic data for repeatable regression tests, then a separate lawful, consented, representative pilot. Do not publish identifiable recordings or use staff voices as a substitute for the intended caller population without a reason.

A small pilot that produces useful evidence

Start with a preregistered acceptance plan. Freeze the model/provider version, prompt, telephony route, locale, parser, field schema, and confirmation design for the test window. Changing any of these mid-test creates a new version or cohort. Write down the ground-truth source and who is allowed to access it. The ground truth should be synthetic or consented and should not appear in public logs.

For each test call, record the scenario ID, expected canonical value, channel conditions, model and integration versions, recognition output, parsed candidate, validation result, confirmation presented, user correction, outgoing payload, and final outcome. Report denominators: number of attempted fields, number of completed fields, number of safe stops, and number of calls excluded. Exclusions must be declared, not removed because a call was inconvenient. If a caller disconnects before confirming, count it as incomplete rather than silently dropping it from the denominator.

Use at least two evaluations. First, a deterministic regression suite that runs after every prompt, parser, model, or integration change. Second, a consented pilot in the actual channel with representative conditions. A fixed synthetic suite helps locate regressions; it does not estimate real-world prevalence. A small live pilot can reveal operational failure modes; it may not have enough data to estimate rare-event rates or compare providers confidently. Neither should be framed as a universal accuracy benchmark.

Set launch gates before seeing results. For high-consequence fields, a single silent mutation or unconfirmed commit should block release until fixed and re-tested. For low-consequence fields, a team may accept some candidate errors if every incorrect value is caught before commit and correction burden remains within its service limit. Define that service limit in advance and include a human escalation threshold. Do not select a threshold after the test to make a preferred vendor pass.

What to publish in a result report

Report the protocol version, date, test environment, data provenance, number of distinct scenarios, number of calls/fields, languages, channels, inclusion/exclusion rules, and any model or prompt changes. Give separate values for exact candidate, exact committed field, safe non-commit, recovery, and correction burden. If you calculate an interval, explain the statistical method and independence assumptions. If the sample is small or hand-selected, say so prominently and do not extrapolate it to all callers.

This article proposes a fifteen-case fixture set; no calls were run and no original error-rate data were collected for it. It therefore reports no measured ASR or voice-agent performance. The numerical example above is hypothetical and explicitly conditional. It should not be cited as a measured finding.

Workflow map

flowchart TD
  A[Caller audio] --> B[Versioned recognition]
  B --> C[Raw hypothesis retained or minimized by policy]
  C --> D[Typed deterministic parser]
  D --> E{Schema and domain validation pass?}
  E -- no --> F[Clarify or offer alternate channel]
  E -- yes --> G[Present candidate in privacy-appropriate form]
  G --> H{Caller confirms this exact candidate?}
  H -- correction --> D
  H -- no or ambiguous --> F
  H -- yes --> I[Freeze confirmed value and action digest]
  I --> J[Revalidate exact outgoing payload]
  J --> K{Payload still matches confirmation?}
  K -- no --> F
  K -- yes --> L[Commit once; record outcome]

The diagram intentionally keeps interpretation and commit apart. Confirmation is scoped to one candidate and one pending action; a changed candidate must return to validation. The diagram does not specify a particular identity or regulatory standard. Those requirements depend on the workflow, jurisdiction, data, and action.

Proof pack: what each source supports

The source types do not agree on a single best confirmation strategy because they answer different questions. Official regulatory context sets scope rather than UI behavior; academic research studies bounded methods; product documentation describes implementation options; benchmark pages report vendor-selected tasks; user posts describe local failures; demos show a selected success path; consulting pages address organization-wide governance. These are complementary evidence, not interchangeable proof. Our recommendation—the typed, versioned, risk-weighted acceptance contract—is an inference from that gap, not a result asserted by any one source.

Decision matrix: how much control should this field get?

“Minimum proposed gate” is a starting design, not legal advice or certification. Assess applicable rules and accessibility needs separately. In some use cases, voice is not an appropriate channel for the field at all.

Failure-mode response table

Every failure should have a defined user-facing status. “We couldn’t safely confirm that, so nothing was changed” is often more honest than pretending the call completed. If the downstream service times out after submission, do not retry blindly; first reconcile whether the action already happened using an idempotency key or a read-only status check. That unknown-outcome problem is separate from speech recognition but part of the same field-integrity contract.

Operational checklist before launch

• [ ] Name every numeric/alphanumeric field and assign a schema, locale, length/range, and canonical representation.
• [ ] State which transformations are allowed; preserve leading zeroes when meaningful.
• [ ] Separate recognition, normalization, validation, confirmation, and commit in logs or trace events.
• [ ] Ensure a language model cannot silently alter a typed value after deterministic parsing.
• [ ] Define the consequence of a wrong value, privacy exposure, reversibility, and escalation route.
• [ ] Design confirmation for the actual user and channel; test masking, interruptibility, DTMF and human alternatives where applicable.
• [ ] Bind confirmation to the current candidate and the exact pending action; invalidate it on correction or payload change.
• [ ] Run the proposed fifteen-case suite plus relevant language, accent, codec, noise, and interruption fixtures.
• [ ] Record exact candidate and exact committed-field results separately from safe stops and recovery.
• [ ] Preregister denominators, exclusions, version changes, and hard release gates.
• [ ] Use synthetic or consented data, restrict access, and state retention/deletion limits.
• [ ] Reconcile uncertain downstream outcomes before retrying a side effect.
• [ ] Give callers a clear stop, correction, and alternate-channel path.
• [ ] Re-test after every material model, prompt, parser, telephony, or tool change.
• [ ] Do not turn a vendor demonstration or headline accuracy percentage into an acceptance certificate.

Bottom line

Voice can be a convenient way to collect structured values, but the workflow should not confuse a fluent conversation with a verified field. Ask the vendor for the exact task, corpus, denominator, channel conditions, and error classes; then run your own versioned acceptance fixtures. For consequential fields, allow no silent mutation and no commit until the caller’s current confirmation matches the exact outgoing value. For low-risk fields, reduce friction only when independent validation and easy correction justify it. In every case, make uncertainty observable, keep the parser deterministic, and give users a safe way to stop.

The proposed suite is a reusable starting point, not benchmark data. Its value is that it makes hidden assumptions testable: what counts as a digit string, which normalizations are allowed, which read-back is sufficient, what happens after a correction, and what happens when confidence never arrives. Those decisions belong in the product contract before the first live call, not in an improvised prompt after a failure.