跳到正文
DEV Community· MarenCrest5138·· 20 小時前AI 評分61

TypeScript 檢索狀態機:法律合約審查中的 4 種失效模式

TypeScript Retrieval State Machine — 4 Failure Modes in Legal Contract Review

AI 導讀

法律合約審查知識庫的 TypeScript 檢索方案以文件狀態機管理索引恢復,並在無法確認來源版本一致時拒絕生成答案。文件分為 pending、verified、stale 或 quarantined 狀態;候選索引須通過區塊數和 contentDigest 驗證,才可透過 manifest 原子發佈。文章亦列出重試、競爭發佈及摘要不符等測試情境。

正文

Short answer: make retrieval recovery a per-document state machine, preserve the last verified index generation, and fail closed when a legal contract cannot be tied to its exact source revision. The deciding constraint is auditability, not raw query uptime. A workspace that quietly serves mixed revisions after a partial ingest is available in the least useful sense.

For an internal knowledge-base bot used during contract review, I would ship four explicit states: pending, verified, stale, and quarantined. Keep generation metadata beside every chunk. A query may read only one verified generation per contract, so recovery never creates a half-old, half-new answer context.

That sounds strict. Good.

Why recovery starts with the document generation

Retrieval-augmented generation separates the model's parametric memory from an external non-parametric memory. That separation is useful, but it also creates two timelines: the contract stored by the workspace and the chunks currently searchable by the retriever. The recovery design has to reconcile them. A retry counter alone can't do that.

Imagine amendment revision 12 produces 86 chunks. The worker writes 61 and loses its lease before verification. Letting those 61 chunks become visible would mix revision 12 with the last complete revision, which can surface an obsolete termination clause beside a current liability clause. The safer unit of publication is the whole generation. Write candidate chunks under a new generation ID, verify the expected count and source digest, then switch one small manifest record from the prior verified generation to the new one. Until that switch, readers stay on the prior version and the UI labels it stale if the source has moved ahead. On retry, the worker uses the same deterministic chunk IDs, checks the 61 existing writes, fills the missing 25, and asks the manifest to publish only if its expected predecessor still matches. If another worker already won, this candidate remains unreachable. Recovery becomes a replay of declared state rather than a guess about which writes landed.

No silent fallback.

This is the concrete constraint that changes the choice: legal reviewers need provenance at paragraph level and deterministic replay after interruption. They don't merely need a vector store that accepts another batch.

I use 409 for a lease conflict in the worker contract because it tells the caller not to publish competing generations. I reserve 422 for a source that can be read but fails contract-level validation, such as a missing document ID. Those are interface choices, not universal protocol rules — the important bit is that each result maps to one state transition and one retry policy.

How should retrieval architecture recover failures in a legal contract review workspace?

Split recovery into four failure classes and make each one visible. Input failures quarantine a document and preserve the last verified generation. Transient execution failures return the candidate to pending with bounded retry metadata. Verification failures discard the candidate from the read path while retaining enough metadata to diagnose it. Read-path failures stop answer generation when no internally consistent generation can be loaded.

Failure class Next state Read behavior
Invalid source metadata quarantined Keep the prior verified generation
Interrupted chunk write pending Keep the prior verified generation
Count or digest mismatch quarantined Exclude the candidate
Missing verified generation pending Refuse answer generation

Don't collapse those cases into failed: true. A single flag makes operators guess whether retrying is safe, whether users are seeing old text, and whether an answer already cited an invalid generation. The state record should carry sourceRevision, generationId, contentDigest, expectedChunks, writtenChunks, attempt, and updatedAt. The answer record should copy generationId plus the retrieved chunk IDs. That gives a reviewer a stable chain from answer to passage to source revision.

The catch is latency. Whole-generation publication uses extra storage during an update, verification delays freshness, and fail-closed reads can temporarily withhold an answer. It is not suitable when partial results are harmless and sub-second freshness matters more than revision consistency. For a low-risk support FAQ, a best-effort incremental index may be the better trade. Legal review flips that priority.

The smallest working TypeScript implementation

The core doesn't need a framework. It needs a narrow storage contract and transitions that can be tested without the embedding model or search engine installed. That keeps config bloat out of the recovery logic.

type Phase = "pending" | "verified" | "stale" | "quarantined";

type Generation = {
  documentId: string;
  sourceRevision: string;
  generationId: string;
  contentDigest: string;
  expectedChunks: number;
  writtenChunks: number;
  attempt: number;
  phase: Phase;
  updatedAt: string;
};

type Chunk = {
  id: string;
  generationId: string;
  text: string;
  sourceOffset: number;
};

interface RetrievalStore {
  putCandidate(record: Generation, chunks: Chunk[]): Promise<void>;
  readCandidate(generationId: string): Promise<Generation | undefined>;
  publish(documentId: string, generationId: string): Promise<boolean>;
  quarantine(generationId: string, reason: string): Promise<void>;
}

export async function verifyAndPublish(
  store: RetrievalStore,
  generationId: string,
): Promise<"published" | "retry" | "quarantined"> {
  const candidate = await store.readCandidate(generationId);
  if (!candidate) return "retry";

  if (!candidate.documentId || !candidate.contentDigest) {
    await store.quarantine(generationId, "invalid-source-metadata");
    return "quarantined";
  }

  if (candidate.writtenChunks !== candidate.expectedChunks) return "retry";

  const published = await store.publish(candidate.documentId, generationId);
  return published ? "published" : "retry";
}

publish must be an atomic compare-and-swap against the manifest version. Two workers may finish the same contract, but only the candidate based on the expected prior manifest wins. The loser can be collected later; it never reaches retrieval. No distributed lock is required in the query path.

Test the ugly transitions. Start with a candidate at 60/86, rerun the same generation, and assert that chunk IDs are idempotent rather than duplicated. Race two complete candidates and assert that one manifest generation wins. Corrupt a digest and assert quarantine. Then request an answer with no verified generation and assert that generation is refused before prompting. These tests matter more than a happy-path similarity score because they prove what users can see during recovery.

One caveat: I'm not sure a fixed retry ceiling fits every document source. The right ceiling depends on measured failure duration and queue pressure. Record both, then set the bound from observed data instead of copying a fashionable exponential-backoff recipe.

What I would change at scale

At larger volume, I would keep the state machine and replace inline verification with an append-only event stream plus a reconciler. The reconciler scans for pending generations whose leases expired, manifests pointing at missing generations, and source revisions newer than their verified index. It emits metrics by phase and age, not one vague error total.

Index cost is the pressure point. Full duplicate generations can become expensive when thousands of long agreements are amended frequently, so retain the atomic manifest while deduplicating chunks by content digest. Reuse is allowed only when normalized text, embedding configuration, and access-control scope match; changing any of those inputs creates a new chunk identity. Measure candidate bytes, verified bytes, reused chunks, verification lag, and queries blocked for consistency. Those five counters show whether recovery is buying trustworthy reads or just accumulating storage.

Keep the old verified generation for a defined rollback window, then garbage-collect unreachable candidates. Don't keep everything forever. Conversely, don't delete the prior generation in the same transaction that publishes the new one; rollback and audit export need a separate retention decision.

Measure first.

There is no universally correct threshold here. A workspace with weekly uploads and strict review deadlines should favor a longer rollback window. A high-volume internal support corpus with reproducible source imports may favor aggressive cleanup and incremental indexing. The selection rule is plain: choose the recovery policy whose failure state the team can explain to a reviewer without reconstructing hidden worker behavior.

References

來源:DEV Community · dev.to