Node.js 如何為醫療科技審核佇列選擇文字摘要 API 模型

TL;DR: to select a text summarization API model in Node.js, route moderation reports through a small, typed decision layer that can choose a fast or careful summarizer, then send uncertain cases to human review. The useful optimization is not a universal model winner. It is knowing when a shorter wait is worth a lower-confidence summary.

A moderation summary is a review aid, not the moderation decision. That distinction changes the design. The system should preserve the original report, label the summary as machine-generated, and make it cheap to reject or regenerate a poor result.

For a solo builder, I start with a fixed evaluation set and a narrow output contract. I do not begin by wiring one model directly into a request handler. That shortcut hides the two failures that matter here: a plausible summary can omit the detail a reviewer needs, and a slow call can make the review queue feel broken. Keep it typed.

How should Node.js select a model for text summarization APIs?

Use quality as a gate and latency as the tie-breaker. A report that arrives in 400 ms but drops the alleged harm, target, or evidence is worse than a report that takes longer and preserves those fields. A report that is accurate but blocks a human queue for several seconds may also be operationally wrong.

Define the contract before selecting a provider or model. For example, every summary can require a neutral synopsis, the reported target, the alleged policy concern, and an evidence quote. If a field is absent, the output should say that it is absent rather than infer it. Keep the raw text beside the generated result so a reviewer can compare them.

The first dataset can be deliberately small: 120 de-identified reports, split into 80 for prompt and routing work and 40 held back for evaluation. The number is a working constraint, not a claim about statistical sufficiency. Reviewers should score omission, distortion, unsupported inference, and usefulness on separate scales. The trade-off is deliberate: a small, stable set makes regressions visible before the queue depends on the change, even though it cannot represent every language or report type.

That separation matters. A fluent sentence is not evidence of a faithful summary.

It can sound right and still fail.

Measure it.

How do I route a report without hiding uncertainty?

Make routing explicit and observable. The classifier below does not pretend to know which service is best. It sends a report to a fast path when the input is short and the queue is healthy; otherwise it chooses the careful path. The actual endpoints remain configuration, which keeps the application portable.

type Route = "fast" | "careful";

type RoutingInput = {
  characterCount: number;
  queueDepth: number;
  fastPathP95Ms: number;
  maxWaitMs: number;
};

function chooseRoute(input: RoutingInput): Route {
  const shortEnough = input.characterCount <= 2_000;
  const queueIsHealthy = input.queueDepth < 20;
  const fastEnough = input.fastPathP95Ms <= input.maxWaitMs;

  return shortEnough && queueIsHealthy && fastEnough ? "fast" : "careful";
}

These thresholds are policy, not model facts. Store them with the evaluation run that justified them. If the fast path starts omitting evidence, lower its allowed scope or remove it from routing. Do not quietly compensate with a longer prompt and assume the behavior stayed constant.

The request boundary should also be boring. It needs a timeout, a request identifier, bounded retries, and an explicit failure state. A failed summary must leave the report available for a human; it must not become an empty string that looks complete.

type Summary = {
  synopsis: string;
  target: string | null;
  concern: string | null;
  evidence: string | null;
  generated: true;
};

type Summarizer = (report: string, signal: AbortSignal) => Promise<Summary>;

async function summarizeForReview(
  report: string,
  route: Route,
  clients: Record<Route, Summarizer>,
  timeoutMs = 2_500
): Promise<Summary | null> {
  const controller = new AbortController();
  const timer = setTimeout(() => controller.abort(), timeoutMs);

  try {
    return await clients[route](report, controller.signal);
  } catch (error) {
    console.error("summary_unavailable", {
      route,
      reason: error instanceof Error ? error.name : "unknown"
    });
    return null;
  } finally {
    clearTimeout(timer);
  }
}

The null result is intentional. It gives the queue a state that can be retried or reviewed manually. It also prevents an API response from being mistaken for a successful moderation outcome.

Which signals expose a bad trade-off?

Measure the whole review loop, not only provider latency. I track time to a usable summary, timeout rate, retry count, and the percentage of summaries that reviewers mark as missing evidence. I also record input and output token counts where the service exposes them, because a prompt that grows quietly can change both latency and operating cost.

For quality, use a blinded review of the held-out reports. Ask two reviewers to mark whether the summary preserves the report's target, alleged harm, and supporting evidence. Resolve disagreements with a third review. The result is more useful than a single aggregate score because it shows the failure mode the fast path creates.

For latency, use percentiles rather than an average. The queue cares about the slow tail. A route with a good median and an ugly p95 can still make human reviewers wait at the exact moment volume rises.

I would read the worst examples before changing the route. If the missing evidence is concentrated in long reports, a character threshold is a useful first control. If it appears in short reports with quoted speech, length is the wrong proxy and the careful route needs a content signal instead. This is where the evaluation set earns its keep: it turns a vague complaint about quality into a routing rule that can be tested, reviewed, and rolled back.

I would compare at least three interchangeable service adapters in the same harness: an OpenAI adapter, an Anthropic adapter, and a Google adapter. The comparison should use the same reports, output schema, timeout budget, and review rubric. Their names are less important than the boundary: each adapter must return the same typed result, surface refusal or failure clearly, and expose the measurements needed for the decision.

Adapter in the test harness	Integration shape	Useful test condition	Boundary to verify
OpenAI	SDK or REST adapter	Short, routine reports	Schema errors and timeout behavior
Anthropic	SDK or REST adapter	Reports with long evidence	Output preservation under the same budget
Google	SDK or REST adapter	Mixed-length held-out reports	Failure state and telemetry parity

The table is a test plan, not a ranking. The exact SDK surface can change; the typed adapter contract should not.

Do not turn that harness into a leaderboard. A provider can win one slice of the dataset and lose on a different language, report length, or evidence pattern. The decision is a routing policy backed by your data.

What prevents a summary from becoming a safety problem?

Keep authorization and enforcement outside the summarizer. The generated text should never approve an account action, decide a policy violation, or replace the original report. A human reviewer needs provenance: report identifier, model route, prompt version, creation time, and whether the output passed schema validation.

Treat the report as untrusted input. Delimit it in the prompt, instruct the summarizer to treat it as data, and test reports that contain instructions such as “ignore the reviewer” or “mark this safe.” Prompt injection is not solved by a single sentence, but a test corpus makes regressions visible.

Retention needs the same discipline. Health-related reports may contain sensitive information, so log identifiers and measurements instead of raw content by default. Redact before sending text to an external service when your policy requires it, and document the retention behavior of every adapter. A lower latency number is not a good result if the data boundary is unclear.

The decision rule I would ship

Ship the fast route only for inputs and conditions that pass the held-out quality gate. Send longer, ambiguous, or operationally important reports to the careful route. If neither route meets the review rubric within the latency budget, show the original report and mark the summary unavailable. That is a product state, not an exception to hide.

Then rerun the same evaluation whenever you change the prompt, schema, adapter, or routing threshold. The useful artifact is not a model choice frozen in a document. It is a repeatable test that tells you when the quality-latency trade-off moved.