RAG 白話解説：校園服務台如何只根據文件回答問題

Internal GPT is a helpdesk I built for college environments. Administrators upload institutional documents, and students and staff ask natural-language questions. The answers must come strictly from those documents, with source citations.

This post explains how Retrieval-Augmented Generation (RAG) works in plain English, using that project as the example. If you've heard the acronym and nodded without being sure what's inside, this is for you.

The problem RAG solves

A language model knows a lot about the world, but nothing about your college's attendance rules, fee deadlines or hostel policy. If you ask it anyway, it may produce a confident, plausible, completely invented answer. In an institution, an invented policy is worse than no answer.

RAG fixes this by separating knowledge from language:

• The documents hold the facts.
• The model turns those facts into a readable answer.

The model never has to remember anything. It just reads what you hand it.

The pipeline

There are two phases: preparing the documents (once) and answering questions (every time).

Phase 1: indexing (when a document is uploaded)
  upload → extract text → split into chunks → embed each chunk → store

Phase 2: answering (when a question is asked)
  question → embed it → find the most similar chunks → build prompt → generate answer + sources

Let's take each step.

1. Extract text

PDFs, Word documents and plain text each need their own extraction. Some documents have complex layouts that confuse naive extractors, so I added a fallback: if the primary method fails, the system tries alternatives before reporting an error.

2. Chunk

You can't embed a 60-page handbook as one unit. The text is split into semantically meaningful pieces, ideally each one covering one idea. Chunk size and overlap matter a lot:

• Too big, and a chunk contains several topics, so it matches many questions weakly.
• Too small, and a chunk loses context ("the fee is due by the 5th" - which fee?).
• A little overlap between chunks stops a sentence from being cut off from its context.

function chunk(text: string, size = 800, overlap = 120): string[] {
  const chunks: string[] = [];
  for (let start = 0; start < text.length; start += size - overlap) {
    chunks.push(text.slice(start, start + size));
  }
  return chunks;
}

(A real chunker splits on paragraph and sentence boundaries first. This is the simplest possible version, to show the idea.)

3. Embed

An embedding turns a piece of text into a list of numbers (a vector) that captures its meaning. Two texts about the same thing end up with vectors that point in similar directions, even if they share no words. "How do I pay the semester fee?" and "Fee payment procedure" land close together.

4. Store

The vectors are stored next to the original text. In the first version I used Supabase with the embeddings in a simple table, and designed the retrieval layer behind a clean abstraction so it can be upgraded to pgvector for larger deployments without touching anything else.

5. Retrieve

When a question arrives, it's embedded the same way, and a similarity search finds the chunks whose vectors are closest:

function cosine(a: number[], b: number[]) {
  let dot = 0, na = 0, nb = 0;
  for (let i = 0; i < a.length; i++) {
    dot += a[i] * b[i]; na += a[i] ** 2; nb += b[i] ** 2;
  }
  return dot / (Math.sqrt(na) * Math.sqrt(nb));
}

const top = chunks
  .map(c => ({ c, score: cosine(queryVec, c.vector) }))
  .sort((x, y) => y.score - x.score)
  .slice(0, 5);

This retrieval step is the crucial constraint. It limits the model to information that actually exists in the knowledge base.

6. Generate

The retrieved chunks go into a carefully written prompt that says, in effect: answer using only this context; if the context doesn't contain the answer, say so.

The most important prompt lesson

You must give the model a graceful way to say "I don't know."

Without that explicit instruction, models tend to produce plausible-sounding but fabricated answers, which is unacceptable in an institutional setting. A prompt shaped like this works far better than "answer the question":

You are a college helpdesk assistant.
Answer ONLY from the context below.
If the context does not contain enough information, reply:
"I couldn't find that in the uploaded documents."
Always cite which document each statement came from.

Context:
{{chunks}}

Question: {{question}}

Where it gets hard

• Document formats. Each needs its own pipeline, and messy layouts cause trouble.
• Chunking strategy. I spent significant effort on chunk sizes, overlap and embedding model choice. They change answer quality more than the model does.
• Citations. An answer without a source is a rumour. Every answer needs to point back to the chunk it used.

Grounding goes further than citations

A later project, Hospitality.ai, pushed this idea harder. It extracts insurance-policy clauses into a validated schema, and every claim comes back with a verbatim quote and a line range. The quote is then re-verified against the source and stamped exact, fuzzy or unverified, and the interface renders all three differently.

The principle is the same: don't just trust the model's claim, check it against the source in code. Anything the document doesn't say goes into a visible "gaps" list instead of being filled with what's "usually" true.

The lesson

Production RAG is as much about retrieval quality as generation quality. A perfect language model is useless if it's searching through bad data. The gap between a good retrieval system and a great one is the gap between answers that are merely relevant and answers that are precisely correct.

Try it yourself

Here's a weekend experiment that needs no framework:

1. Take five of your own notes or a short PDF.
2. Chunk them by paragraph.
3. Embed each chunk with any embedding API.
4. Ask five questions and print the top three chunks for each.
5. Notice the wrong retrievals and ask why.

Seeing a wrong retrieval teaches more than reading about it. Fix it by changing chunk size, then re-run.

Debugging a RAG system: symptoms and causes

When answers are bad, resist the urge to blame the model first. Work through this table from the top.

The first row is the most important habit: always look at what was retrieved. Most RAG bugs are visible there.

Measure retrieval, not just vibes

Make a small test set: 20 questions, each with the document you expect it to come from. Then measure how often the right chunk appears in the top results:

type Case = { question: string; expectedDoc: string };

async function hitRate(cases: Case[], k = 5) {
  let hits = 0;
  for (const c of cases) {
    const top = await retrieve(c.question, k);        // your retrieval function
    if (top.some(chunk => chunk.docId === c.expectedDoc)) hits++;
  }
  return hits / cases.length;
}

A "hit rate at 5" of 0.9 means that for 90% of test questions, the right document was in the top five. Change one thing (chunk size, overlap, embedding model), re-run, and keep the change only if the number improves. This is the same principle as the rest of the series: measure what you only believe.

Prompt injection: a quiet risk

Uploaded documents are data, but a model may treat text inside them as instructions ("ignore previous instructions and..."). Basic defences:

• Put document text inside clearly delimited blocks and tell the model it's untrusted data.
• Give the model no tools it could be tricked into using.
• Validate its output against a schema where possible.
• Never let retrieved text decide permissions.

Glossary

• Chunk: a slice of a document sized for retrieval.
• Overlap: text repeated between neighbouring chunks to preserve context.
• Vector / embedding: numeric representation of meaning.
• Cosine similarity: a measure of how aligned two vectors are.
• Top-k: the k most similar chunks.
• Grounding: tying an answer to a specific source.
• Hallucination: a confident answer that isn't supported by the source.

Where to go next

Try adding citations with quotes. Ask the model to return, for each claim, the exact sentence it relied on, then check in code that the sentence exists in the retrieved chunks. Anything that doesn't match is dropped. That one change moves a RAG system from "sounds right" to "provably sourced".

What would you build a document-grounded assistant for? And if you've used pgvector, what did you wish you'd known first?