醫療科技知識庫如何在 Node.js API 批次工作中以低成本 LLM 文字訊號分類

The useful answer is to treat bulk LLM tagging as an incident-detection problem, not as a loop around an API call. For a private health knowledge base, a worker should classify bounded chunks, validate every returned label, record the source row and policy version, and page on a rising rate of unusable results rather than on raw request volume. That design spends a little latency on evidence and retries, because a fast wrong tag can hide the clinical document an operator needs.

The application here is narrow: tagging documents and question-answer candidates in a healthtech knowledge base before they become searchable. Labels might describe document type, care setting, or whether a passage is suitable for retrieval. They are operational metadata, not medical advice. I would keep the model behind a generic interface, use a queue with an explicit retry budget, and make the alert explain which page fired. A dashboard that merely turns green after a batch completes is not an incident response system.

How should a Node.js API handle cheap LLM text classification?

Start with the page an on-call engineer can act on. A batch has completed, but 6.8% of its rows have no valid label while median request latency is normal. That is a quality incident. A different batch has valid labels, but its oldest queued item is 14 minutes old against a 5-minute freshness objective. That is a latency incident. Combining those into one job_failed metric loses the decision.

The queue must tell the truth.

The first signal should be a small set of counters and gauges tied to a batch identifier: accepted rows, rejected rows, retryable failures, permanent failures, queue age, and validation failures. The alert should include the knowledge-base partition, schema version, and policy version. It should not include document contents; private health data belongs behind normal access controls, including in incident tooling.

I initially expected request errors to be the loudest signal. They were easy to count. The more consequential signal is often a successful response that cannot safely be used: an unknown enum value, a label for the wrong document, or a response that exceeds the allowed evidence window. Validation belongs in the worker, before acknowledgement, and the quality threshold must be measured against a reviewed sample instead of guessed from transport status. That sample should be stratified by source and document type, retained under the same access controls as the knowledge base, and reviewed after prompt or schema changes. Otherwise a stable transport metric can conceal a new class of bad tags until retrieval quality has already moved.

A practical alert rule has two gates. Page when the rolling invalid-label rate crosses the agreed threshold and the sample count is large enough to avoid a one-row alert. Page separately when queue age threatens the latency objective. Route both pages to the same runbook, but keep the remediation paths distinct: investigate prompt or schema drift for the first, and capacity, backpressure, or dependency latency for the second.

What does a batch worker need to prove?

Every output should be traceable without storing more sensitive text than the team needs. The minimum useful record is a stable source identifier, a batch identifier, an attempt number, the classifier policy version, the schema version, a timestamp, the result state, and a digest of the input. A digest is not a substitute for authorization or encryption; it is only a way to correlate events without copying the source passage into a broad log stream.

The worker also needs an idempotency boundary. A retry after a timeout must not create a second active tag for the same source revision. A deterministic key such as source_id:revision:policy_version lets storage accept the same completed result twice without treating it as two classifications. If the policy changes, that is a new classification deliberately, not an accidental duplicate.

Duplicates are an incident too.

The code below shows the control shape. It is intentionally generic: the important decisions are the bounded request, the validation step, and the acknowledgement order. The endpoint and model are configuration, not a vendor recommendation.

type Label string

const (
    LabelPolicy         Label = "policy"
    LabelClinical       Label = "clinical"
    LabelAdministrative Label = "administrative"
)

type Classification struct {
    SourceID string `json:"source_id"`
    Revision string `json:"revision"`
    Label    Label  `json:"label"`
    Evidence string `json:"evidence"`
}

func valid(c Classification, allowed map[Label]bool, maxEvidence int) error {
    if c.SourceID == "" || c.Revision == "" {
        return fmt.Errorf("missing identity")
    }
    if !allowed[c.Label] {
        return fmt.Errorf("unknown label %q", c.Label)
    }
    if len(c.Evidence) > maxEvidence {
        return fmt.Errorf("evidence exceeds bound")
    }
    return nil
}

The worker acknowledges a queue item only after the result is valid and durably recorded. A retryable transport error returns the item to the queue with a bounded attempt count. A schema violation is not made healthy by retrying forever; it becomes a visible permanent failure with enough metadata for sampled review. This distinction keeps the retry queue from becoming a second, less visible backlog.

How should quality and latency share one budget?

Quality and latency are coupled, but they are not interchangeable. A private knowledge base can tolerate a slower enrichment batch if the resulting tags are trustworthy and the user-facing retrieval path stays within its objective. It cannot quietly trade correctness for throughput when a label controls which documents are eligible for retrieval.

I use three measurements for this boundary: a reviewed precision estimate for each high-impact label, the age of the oldest unprocessed source revision, and the fraction of rows that reach a fallback state. The first protects meaning, the second protects freshness, and the third exposes degradation that transport metrics miss. Set separate thresholds with the owning team; do not derive a quality target from dependency latency or an arbitrary completion percentage.

A small, stratified review set is more useful than a single global score. Sample by document type, source system, and label frequency. Include hard negatives: passages that mention a clinical term but belong to an administrative policy, and documents whose revision changed without changing their title. Those are the cases that make a clean aggregate metric lie. When an alert fires, preserve the sampled decision, the policy version, the schema version, and the queue age together. That gives the responder a bounded comparison between the last accepted run and the current run. It also avoids the familiar late-night mistake of changing a threshold first and asking what changed second.

For latency, record queue wait and processing time separately. A worker can have a healthy request median while its queue grows because concurrency is capped or retries occupy all slots. Raising concurrency can reduce wait while increasing validation failures if the dependency or prompt is sensitive to load. The decision is which part of the path owns the delay and whether the quality sample moves with it.

What happens when the signal is wrong?

False positives have a cost. If every transient timeout pages, the on-call learns to mute the channel. If a classifier returns a syntactically valid but semantically poor label and no page fires, the incident may surface only when a user cannot find a document. Both failures damage trust, though only one is visible in request logs.

The threshold should be tested like code. Replay a fixed, access-controlled fixture through the validator. Inject timeouts, malformed JSON, unknown labels, duplicate deliveries, and a slow queue. Verify that each case produces the intended state transition and alert annotation. Keep the fixture small enough for CI and representative enough to include ambiguous health-policy examples. Do not put real patient data in it.

A useful state machine has fewer states than most teams start with: pending, processing, accepted, retryable, and rejected. The transition from processing to accepted requires validation and durable storage. The transition to retryable requires an attempt count and a next-attempt time. rejected requires a reason code that can be aggregated without exposing content. This is boring by design. Boring state is easier to page on at 3am.

type Metrics interface {
    Count(name string, labels map[string]string)
    Gauge(name string, value float64, labels map[string]string)
}

func finish(item Item, c Classification, store Store, metrics Metrics) error {
    if err := valid(c, item.Allowed, item.MaxEvidence); err != nil {
        metrics.Count("classification_rejected", map[string]string{
            "reason": "validation",
            "policy": item.PolicyVersion,
        })
        return item.Reject(err)
    }

    key := item.SourceID + ":" + item.Revision + ":" + item.PolicyVersion
    if err := store.PutIfAbsent(key, c); err != nil {
        return item.Retry(err)
    }

    metrics.Count("classification_accepted", map[string]string{
        "policy": item.PolicyVersion,
    })
    return item.Ack()
}

The invariant is not “the API returned 200.” It is “the result passed the contract, the write was durable, and the queue item was acknowledged exactly after that.” Never label metrics with a raw source ID, user ID, or document title.

Where do common tools stop being the answer?

A reranking service, a speech recognizer, and a general language-model endpoint solve different parts of an information workflow. A reranker orders candidates; it does not establish that a source document has the right business label. A speech recognizer turns audio into text; it does not validate a health-policy taxonomy. A general model endpoint may produce a classification, but the queue, schema, review sample, and alert policy remain the application team's responsibility.

That boundary matters more than a feature checklist. The same worker can sit behind different model providers or a self-hosted model if its interface preserves bounded inputs, structured outputs, idempotency, and observable failure states. A product comparison that ignores those controls answers the wrong question.

For a production choice, compare providers only after writing down four tests: can output be constrained to the required schema, can the team measure quality on its own domain sample, can the service meet the queue-age objective under expected concurrency, and can sensitive data follow the approved handling path? Unknown answers belong in the launch decision and runbook. They should not be hidden behind a low request price or a cheerful dashboard.

The decision rule I would leave on the runbook

Choose the fastest path that keeps reviewed label quality above the domain threshold, queue age below the freshness objective, and an auditable record for every accepted result. If latency rises but quality remains stable, schedule the batch with an explicit freshness trade-off. If quality falls while transport metrics look normal, stop promoting new labels, retain source revisions for review, and page on validation and sample quality.

That rule survives a change of endpoint, model, queue library, and deployment shape. More importantly, it tells the on-call engineer why the page fired and what evidence to collect before changing a threshold.