如何以摘要 API 批次處理多份文件並控制租戶成本

Short answer: use a tenant-aware batch to process multiple documents for summarization, preserve one accounting record per document, and retry only operations with a clear idempotency boundary.

Support-ticket summarization is a small data pipeline, not a loop around a summarization endpoint. For a developer-tools team, the useful unit is a tenant-aware batch: collect a bounded set of documents, normalize them, submit work with an idempotency key, record usage before the result is displayed, and make partial failure visible. The summary is only one output. The other is an accounting trail that explains which tenant consumed which amount of work.

That ordering matters. If I let a retry create a second charge-shaped event, or if I attach usage to the worker instead of the tenant, triage gets faster while the bill becomes impossible to explain. I would rather lose one optional summary than lose the ability to answer, "Which customer caused this usage, and can we reproduce the count?"

The ledger is the boundary.

Keep it boring.

It also forces a useful compromise: source text can expire while hashes and usage records remain available for reconciliation.

What is the bill actually made of?

The dominant term in a multi-document job is usually the amount of text sent for processing, not the number of Python function calls. The exact accounting unit depends on the runtime contract, so treat usage as a measured value returned by the processing system or a deliberately defined estimate. Do not infer it from document count. Five short tickets and five exported logs are both five documents, but they are not the same workload.

For each document, keep the tenant identifier, document identifier, content hash, input measurement, output measurement when available, processing status, and request id. A batch record stores requested, accepted, completed, and failed counts plus aggregate usage. The raw text belongs in controlled storage; the cost ledger does not need to duplicate it.

Here is a deliberately boring manifest. Boring records are easier to audit six weeks later.

from dataclasses import dataclass
from typing import Iterable


@dataclass(frozen=True)
class Document:
    tenant_id: str
    document_id: str
    text: str
    content_hash: str


def build_manifest(documents: Iterable[Document], batch_id: str) -> dict:
    items = []
    for document in documents:
        if not document.text.strip():
            continue
        items.append({
            "batch_id": batch_id,
            "tenant_id": document.tenant_id,
            "document_id": document.document_id,
            "content_hash": document.content_hash,
            "status": "pending",
        })
    return {"batch_id": batch_id, "items": items}

I keep the manifest and usage event longer than the generated prose, subject to privacy and contractual requirements. That lets a later investigation reconstruct what was attempted without keeping every customer sentence forever. Deleting source text makes content-level debugging harder, so the retention policy needs an owner and a reason.

How can a batch process multiple documents through a summarization API?

A batch worker should not treat the input array as a transaction. One malformed export, timeout, or rate-limit response should move one item to a retryable or terminal state while the rest continues. The batch summary should say so plainly.

I model item states as pending, running, succeeded, retryable, and failed. Each transition includes an attempt number and timestamp. A retry is allowed only when the previous attempt is known to have ended without a committed result, or when an idempotency key protects the boundary. Otherwise, a network ambiguity can produce duplicate summaries and duplicate ledger events. Three attempts is a policy choice in the example, not a guarantee from the runtime.

import time
from collections.abc import Callable


def process_batch(items: list[dict], summarize: Callable[[str], dict],
                  max_attempts: int = 3) -> list[dict]:
    results = []
    for item in items:
        for attempt in range(1, max_attempts + 1):
            item["attempt"] = attempt
            item["status"] = "running"
            try:
                response = summarize(item["text"])
                item["status"] = "succeeded"
                item["usage"] = response.get("usage")
                item["summary"] = response["summary"]
                break
            except TimeoutError:
                item["status"] = "retryable"
                if attempt < max_attempts:
                    time.sleep(2 ** (attempt - 1))
            except ValueError as error:
                item["status"] = "failed"
                item["error"] = str(error)
                break
        results.append(item)
    return results

This example leaves concurrency out on purpose. Add a worker pool only after measuring service limits and tenant fairness. A global concurrency setting can let one large tenant occupy every slot while smaller tenants wait. Per-tenant quotas, a shared queue, and a maximum batch size make behavior explainable. Add jitter to production backoff so workers do not retry on the same schedule. The limitation is deliberate: this worker is unsuitable as a complete queue, scheduler, or cost allocator without durable state around it.

The first version I would ship has one summary per document. Combining documents can reduce request overhead, but it weakens attribution: a bad result becomes harder to map to a source ticket, and one oversized document can prevent the whole group from completing. Grouping is an optimization, not the default correctness boundary.

Which question should the operational record answer?

The useful question is not only "did the API return 200?" It is "can support explain this result and this usage to one tenant?" A successful transport response can still contain an unusable payload, a truncated result, or no usage information. Validate the response schema, cap accepted output length, and store the request id beside the result.

Emit structured events such as summary.requested, summary.completed, and summary.failed. Include tenant id, batch id, document id, attempt, status, and measured usage where available. Do not put ticket text in ordinary logs. Large payloads make the signal difficult to scan and create a privacy problem.

A dashboard should show batches by status, failures by reason, latency by document size, and usage by tenant. The last view is the decision axis. An aggregate total hides noisy neighbors; a tenant view makes an unexpectedly expensive integration visible before the monthly report becomes a surprise.

If a worker times out after the remote operation accepted the request, local state is uncertain. Mark it unknown, reconcile it using the request id, and do not silently classify it as a free failure. Treating uncertainty as zero is how cost reports drift.

What belongs in the deployment and test boundary?

Tests should cover decisions, not only the happy-path parser. Feed the worker an empty document, duplicate content hash, malformed response, timeout after acceptance, and a tenant that reaches its quota. Assert both item state and ledger event. A unit test that checks only summary misses the failure that matters to finance and support.

Use deterministic fixtures for summarization responses. Keep a small integration test for authentication, request serialization, response validation, and usage extraction. Production customer text does not belong in a test fixture.

Before release, set limits for document size, batch size, retries, queue age, and retained source content. These are policy settings, not magic constants scattered through workers. Include effective values in batch metadata so an operator can distinguish a code change from a changed limit.

The API boundary also needs bounded timeouts, explicit non-success handling, and correlation identifiers. WebRTC is not a summarization protocol, but MDN's WebRTC API documentation illustrates the useful lifecycle principle: model distinct objects and states instead of treating a network operation as one opaque call. Apply that to the backend batch: request, processing state, result, and cleanup are separate points.

The retention choice is product behavior

A support-triage system is judged by the speed of the first useful answer, but it is operated by people who handle privacy requests, tenant disputes, and unreliable networks. Keep what the next investigation needs. My default is a manifest, hashes, state transitions, usage records, and a pointer to controlled source storage; raw text expires according to policy.

That choice costs diagnostic detail. If a summary looks wrong after the source expires, the team may prove which document was processed but not recreate its wording. Indefinite content retention creates a different risk. The right answer is a written retention period, an access audit, and a redaction path.

The practical decision rule is simple: batch only work that can share a failure policy and an accounting boundary. Preserve per-document identity when transport groups requests. Report measured usage to the tenant that submitted the document. Reconcile ambiguous outcomes. Make partial completion a first-class result.

That is how a summarization API becomes useful infrastructure for support triage: the prose helps the queue, while the ledger makes the system operable.