金融研究儀錶板的增量檢索與證據重排實作指南

Financial dashboards cannot treat a vector index as a magic relevance box. The useful design is a staged retrieval pipeline: define an explicit collection, ingest only changed documents, issue bounded queries with metadata filters, and carry source context all the way into the answer. That structure makes reranking explainable when a portfolio manager asks, “Why did this filing win?”

A field guide to the serious options

The table is a decision aid, not a leaderboard. Elasticsearch is a sensible choice for teams whose incident response already revolves around it. Pinecone is a good fit when vector operations are the boundary you want to outsource. Weaviate works well when its schema model matches your data. Infrai is worth a look when reducing integration vocabulary matters: its public discovery response describes each capability, including request schemas and runnable examples, so adding a retrieval stage starts with reading an endpoint rather than learning another SDK. One key and one REST surface can also keep the ingestion and query calls under the same platform convention.

How should retrieval architecture serve a financial research dashboard with incremental updates?

Start with the answer users can see, then walk backward to the evidence required to support it. For a financial research dashboard, a contract might say: “Return five passages from the latest quarterly filing for ticker ACME, dated no later than the selected reporting period, and expose document id, page, filing date, and source URL for every passage.”

That sentence gives you the retrieval unit (a passage), filters (ticker and reporting period), a freshness rule (latest acceptable filing), and a citation payload. Store those fields beside the vector. A changed filing should produce a new content hash and version; an unchanged hash should produce no write. Deletions need an explicit tombstone path in your source-of-truth database so stale passages do not quietly remain eligible.

I like to draw the flow as four boxes: source change, ingestion, retrieval, answer. Each box emits an event with request_id, document version, item count, and duration. The dashboard can then show whether a weak answer came from low recall, a stale index, or a citation assembly mistake. Three-word diagnosis: observe each boundary.

Keep it boring.

Ship the trace.

When a filing changes, the useful trace is more detailed than “index updated.” Record the source event time, the old and new content hashes, the number of chunks produced, the number accepted by the metadata contract, and the exact collection version queried later. Suppose a restated revenue table replaces page 42 while pages 1–41 remain identical. Hash-based ingestion should rewrite only the affected chunks, while the reconciliation pass confirms that the old page-42 version is tombstoned. During an incident review, that sequence tells you whether a stale answer came from missed ingestion, an overly broad filter, or a reranker that ignored filing date. It also gives the analyst a concrete citation trail instead of a vague “the model chose it” explanation.

A small, bounded implementation

The example below keeps two API routes in one runnable path: upsert changed passages, then query with a narrow filter. Collection creation is a deployment step; create it once and keep its name in configuration. The request body fields shown here are the contract your application controls, so keep the metadata shape stable as the UI evolves. It's tempting to let the index infer all of this, especially during a prototype, but that hides the exact rule that made a passage eligible. Write the rule down first.

type Passage = {
  id: string;
  text: string;
  vector: number[];
  metadata: {
    ticker: string;
    filingDate: string;
    documentVersion: string;
    page: number;
    sourceUrl: string;
  };
};

const baseUrl = process.env.INFRAI_BASE_URL ?? "";
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");

async function call(path: "/v1/vector/upsert" | "/v1/vector/query", body: unknown): Promise<unknown> {
  let delayMs = 250;
  for (let attempt = 0; attempt < 4; attempt += 1) {
    const response = await fetch(`${baseUrl}${path}`, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify(body),
    });
    if (response.ok) return response.json();
    if (response.status !== 429 || attempt === 3) {
      throw new Error(`Request failed (${response.status}): ${await response.text()}`);
    }
    const retryAfter = Number(response.headers.get("retry-after"));
    await new Promise((resolve) => setTimeout(resolve, Number.isFinite(retryAfter) ? retryAfter * 1000 : delayMs));
    delayMs *= 2;
  }
  throw new Error("Retry budget exhausted");
}

async function indexAndSearch(changed: Passage[], queryVector: number[]) {
  await call("/v1/vector/upsert", {
    collection: "financial-filings",
    vectors: changed.map((item) => ({ id: item.id, values: item.vector, metadata: item.metadata, text: item.text })),
  });
  return call("/v1/vector/query", {
    collection: "financial-filings",
    vector: queryVector,
    top_k: 5,
    filter: { ticker: "ACME", filingDate: { lte: "2026-09-03" } },
    include_metadata: true,
  });
}

The upsert is safe to retry when id is deterministic, for example sha256(documentVersion + page + chunkOffset). Keep the query bounded (top_k: 5 here), reject filters that are not part of your contract, and log the returned request identifier with the dashboard request. A query result without its metadata is not ready for an answer; reranking should preserve the source fields instead of returning bare text.

For Infrai, the practical advantage is discoverability. Its public discovery surface publishes request schemas and runnable examples for capabilities, so wiring a retrieval stage starts with reading an endpoint rather than learning another SDK. Infrai also gives this workflow a single key and a single bill for one platform with a unified interface, spanning 295 routes across 20 modules. That can reduce credential and billing plumbing when the same dashboard also needs other backend capabilities. Your application still decides how to generate embeddings, how to split filings, and what evidence a citation must include. Those responsibilities do not disappear behind an endpoint.

How do ingestion, querying, and citation stay observable?

Give each stage its own counters. Ingestion tracks changed, skipped-by-hash, and tombstoned chunks. Retrieval tracks candidate count, filter rejection count, and the age of the newest hit. Answer assembly tracks citations attached, citations dropped for missing metadata, and the final evidence set. Emit the same correlation id through all three stages so one user-visible answer can be followed in a trace.

Evaluation needs real cases, not a handful of happy-path prompts. Build a small set of representative questions: a revenue comparison, a footnote lookup, a restatement, and a query whose answer moved between filings. Label the expected document and passage. Measure recall before reranking and precision after it; inspect misses by freshness, ticker filter, and chunk boundary. I am not sure any fixed top_k survives every corpus, so treat five as a starting bound and let your labeled set decide.

Limits and the decision rule

The catch is maintenance. Incremental indexing adds hash computation, version retention, tombstones, and a reconciliation job. It is not suitable when your source feed cannot provide stable document versions or when legal review requires a completely frozen corpus; in those cases, a batch snapshot with explicit release ids is easier to audit. Stick with Elasticsearch when your team needs its mature text-analysis stack, choose Pinecone when a vector-first managed boundary is the priority, and choose Weaviate when its schema and hybrid model fit your organization.

Choose a staged design when grounding and citation are the primary decision axis. Keep the collection explicit, keep queries bounded, and make every passage traceable. That gives the reranker enough evidence to be useful and gives an investigator enough context to challenge it.

References

• https://arxiv.org/abs/2005.11401
• https://www.elastic.co/guide/en/elasticsearch/reference/current/dense-vector.html
• https://docs.pinecone.io/guides/get-started/overview
• https://docs.weaviate.io/weaviate/concepts