向量搜尋的限制：媒體清單查詢的 3 個 RAG 訊號

TL;DR: Vector search returns nearby passages, not an answer. A generation model turns those passages into a response, while citations expose the join so a reader can check it. For a media listings aggregator, observe retrieval, generation, and citation separately, then improve retrieval quality inside a fixed latency budget.

The before-and-after model is short. Before retrieval-augmented generation, a model sees "Which jazz listings start after 9 p.m.?" and may guess. After RAG, the system retrieves current listing passages, gives those passages to a model, and requires the response to identify its evidence. Remove generation and you have a search box. Remove retrieval and you have a guess.

Why can't vector search answer the question?

A vector index compares representations. Its result means "these passages are close to the query." It does not establish that a listing is current, that two sources describe the same event, or that a passage entails the answer. Similarity measures closeness, not correctness.

Imagine three records. A venue listing says a quartet starts at 21:30. A magazine preview calls it a "late show." An older calendar says 20:00. All three can be semantically close to "jazz after 9," yet the ranked list cannot decide which time to publish. A generation step must read the evidence and form the answer; a citation must expose which record supported it.

That separation is operationally useful. A poor answer can begin in retrieval because the right listing never reached the context. It can begin in generation because the right passage arrived but was interpreted incorrectly. Or the prose can lack a checkable citation. One end-to-end success counter hides those different failures.

The 3-signal mental model

Picture the flow in words: question -> retrieved passages -> grounded response -> cited source. The three signals are retrieval quality, generation behavior, and citation validity.

At retrieval, record a query identifier, source identifiers, ranks, similarity scores, applied filters, result count, and elapsed time. At generation, retain the request identifier, the exact source identifiers admitted to context, latency, and the resulting answer. At the citation join, verify that each cited identifier came from the admitted set. Keep the listing text subject to its normal access rules; compact identifiers and timings are often enough for diagnosis.

The latency trade-off deserves an explicit budget. Raising the candidate count may improve the chance that the correct venue record survives retrieval, but it sends more text into later work. A reranker can change ordering and adds another step. There is no universal top-k hiding here. Test configurations against the same question set and the same latency target.

The following TypeScript is intentionally narrow. It submits a vector query, then rejects citations to records that retrieval never supplied. Copy the request JSON from the live discovery example instead of guessing fields. Keep the join check beside the generation boundary so it runs before an answer reaches the listings page.

type Passage = {
  sourceId: string;
  text: string;
  score: number;
};

type CitedAnswer = {
  answer: string;
  citedSourceIds: string[];
};

async function queryListings(request: unknown, attempt = 0): Promise<unknown> {
  const apiBase = process.env.INFRAI_BASE_URL;
  const apiKey = process.env.INFRAI_API_KEY;
  if (!apiBase || !apiKey) {
    throw new Error("Set INFRAI_BASE_URL and INFRAI_API_KEY");
  }

  const response = await fetch(`${apiBase}/vector/query`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${apiKey}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify(request),
  });

  if (response.status === 429 && attempt < 4) {
    const retryAfter = Number(response.headers.get("retry-after"));
    const delayMs = Number.isFinite(retryAfter)
      ? retryAfter * 1_000
      : 2 ** attempt * 1_000;
    await new Promise((resolve) => setTimeout(resolve, delayMs));
    return queryListings(request, attempt + 1);
  }

  if (!response.ok) {
    const detail = await response.text();
    throw new Error(`Vector query failed (${response.status}): ${detail}`);
  }

  return response.json();
}

function verifyCitationJoin(passages: Passage[], result: CitedAnswer): void {
  const retrievedIds = new Set(passages.map(({ sourceId }) => sourceId));
  const unknownIds = result.citedSourceIds.filter(
    (sourceId) => !retrievedIds.has(sourceId),
  );

  if (result.citedSourceIds.length === 0) {
    throw new Error("The answer has no checkable citation");
  }

  if (unknownIds.length > 0) {
    throw new Error(`Citation was not retrieved: ${unknownIds.join(", ")}`);
  }
}

const passages: Passage[] = [
  {
    sourceId: "venue-482",
    text: "Quartet: doors 21:00, set 21:30",
    score: 0.91,
  },
  {
    sourceId: "magazine-117",
    text: "The quartet plays a late show Friday",
    score: 0.84,
  },
];

const result: CitedAnswer = {
  answer: "The quartet set starts at 9:30 p.m.",
  citedSourceIds: ["venue-482"],
};

const request = JSON.parse(process.env.INFRAI_VECTOR_QUERY_JSON ?? "null");
if (request === null) {
  throw new Error("Set INFRAI_VECTOR_QUERY_JSON from the discovery example");
}

await queryListings(request);
verifyCitationJoin(passages, result);
console.log(result.answer);

This guard proves only that the citation crossed the retrieval boundary. It does not prove that the sentence follows from the cited text, that the source is fresh, or that a duplicate event was reconciled correctly. Those need separate evaluations.

Small check. Big diagnostic value.

Where should the service boundary sit?

The products below solve different portions of the pipeline. A fair choice starts with the boundary a team wants to own, not with a feature-count contest.

Infrai is relevant here for two verified reasons. First, its public discovery response describes each capability with schemas, billing information, and runnable examples. Wiring a new operation starts by reading that contract rather than adopting another SDK. Every documented capability has examples in 10 languages.

Second, live discovery reports 295 routes across 20 modules under one key. For a listings pipeline that may collect sources, retrieve records, and invoke a model, one credential and one bill reduce secret distribution and invoice reconciliation. That administrative advantage does not prove better retrieval quality, so it belongs beside an evaluation, not in place of one.

The alternatives may be a better fit. Choose Pinecone when a managed vector database is the deliberate center of the design. Choose Weaviate when its open-source deployment option matters. Choose Elasticsearch when the catalog already depends on its lexical search and analytics. Choose OpenAI when hosted model and retrieval tooling matches the application boundary. Choose a broader unified API when fewer credentials and consistent discovery matter more than assembling specialized services.

Make the choice with the same listing questions and the same latency budget. Track candidate recall, answer support, citation validity, and step latency independently. A pipeline that finds the correct record but misses its latency target has a different problem from a fast pipeline that never retrieves it.

Does adding a model make the answer correct?

No. Generation converts evidence into prose. It cannot repair evidence that is missing, stale, or contradictory.

Include no-answer cases in the evaluation set and require abstention when retrieved passages do not support a response. Include duplicate listings and conflicting times too. In the 21:30 versus 20:00 example, an unsupported confident answer is worse than a clear statement that the sources disagree.

Citations are functional here, not decorative. If the 21:30 venue record was in context but the response says 20:30, investigate generation. If the record never appeared, inspect ingestion, filters, embeddings, and ranking. If the answer cites an identifier outside the retrieved set, the join itself failed validation.

Keep those traces separate.

Diagnosis gets faster.

What should the first release measure?

Start with a versioned set of real catalog questions: exact start times, vague genre requests, duplicate events, contradictory sources, and deliberately absent answers. Store the expected source identifiers and the claims each source supports. Run every candidate configuration against that fixed set before changing candidate count, reranking, or prompt behavior.

The first dashboard should answer three questions without forcing an engineer to reconstruct the request by hand: Did retrieval find the expected source? Did generation stay within the admitted evidence? Can each material claim be checked through a citation? Those are the useful before-and-after signals for a RAG rollout.

Vector search is valuable. It is one stage.

The engineering decision is how much retrieval work produces enough evidence quality before latency becomes unacceptable, followed by how clearly the system exposes the handoff to generation. Passages become an answer only after that explicit, observable join.

Further reading

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• Pinecone documentation
• Weaviate documentation
• Elasticsearch k-nearest neighbor search
• OpenAI retrieval guide