LLM 與 RAG 實作：PDF 文件檢索流程

What you need first ?

Load PDF
Split PDF --> The output is Chunks.
Build Vector DB ( Input will be CHUNKS ) --> Also need instance or object to Query.
Retriever
Augment or Build Prompt
Generate_answer --> ask the LLM --> Groq
RAQ Query

Things Needed

Embedding --> nomic-embed-text ( Model )

Install Ollama

>llama --version
ollama version is 0.40.6

> ollama list
> ollama pull nomic-embed-text

Test with the below curl URL if you are getting the response.

what is this nomic-embed-text Model --> Its just a Weight , to execute this we have some tools like Ollama , lama.cpp , etc

Another part of LLM

LLM for retrieval via Groq ( Chatgpt )

2 LLM

1 local embedding model
Another 1 , LLM model via Groq

Lets Begin

Load the Env details

Env variable

Load the PDF

PyPDFLoader
This loader will take care all the things.

Set the env and run requirement txt

python3 -m venv .venv
source .venv/bin/activate
(.venv) pip install -r requirements.txt
pip freeze > requirements.txt
python3 -m pip install pypdf

Run and check the PDF Part

This code will read the PDF and give the details.
It converts the entire PDF into text.

You can see many Chinese characters , this needs to be cleaned and sent for Chunking.

Clean the docs

Write one more function to clean the documents.
The complete document is in list. It will be like each page.

def clean_docs(documents):
    """Normalizes whitespace, removes null bytes, and sanitizes text encoding for each document."""
    cleaned_docs = []
    for doc in documents:
        # Handle objects with page_content attributes vs raw text strings
        if hasattr(doc, "page_content"):
            text = doc.page_content
        else:
            text = str(doc)

        # Replace multiple whitespace characters with a single space
        text = re.sub(r"\s+", " ", text)
        # Remove null control characters
        text = re.sub(r"\x00", "", text)
        # Drop invalid UTF-8 sequences and decode back to a clean string
        text = text.encode("utf-8", "ignore").decode("utf-8")
        text = text.strip()

        # Update the document object or append the cleaned text
        if hasattr(doc, "page_content"):
            doc.page_content = text
            cleaned_docs.append(doc)
        else:
            cleaned_docs.append(text)
    return cleaned_docs

Check for the 100th page

Comment the line and check if getting after the cleaning

Change the function like below

Now we can read the PDF. i.e load the PDF got completed.

Chunking Part

function split , its a text splitter.

The above one is a CHUNK
Now another CHUNK - Its in yellow color. ( with a overlap )
" Such as ... " is a overlap. , it will be both the CHUNK.
Overlap means each chunk repeats the last 150 characters of the one before it. If a sentence is cut in half at the edge, it still appears whole in one of the chunks.

This is called - Recursive character text splitter, langchain already implemented this.

from langchain_text_splitters import RecursiveCharacterTextSplitter

Need to provide the CHUNK size and also the overlap.
If the PDF as diagram this code won't work.

def split_pdf(documents):
    """Splits documents into smaller overlapping chunks to optimize for embedding and retrieval."""
    splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=150)
    return splitter.split_documents(documents)

Now vector DB

The input to this is " CHUNKS ".

Embeddings
For embedding , we are going to use Ollama Model which is in the local. For this Ollama model , langchain is supporting.

from langchain_ollama import OllamaEmbeddings

Persist directory is used.
Return the Vector Store.

DB's

Sqlite3 --> Its a Document DB --> No need to install like Postrgres , its just a python package . ( DB browser for SQLITE )
In-memory DB's. [ TBD ]

Initial Setup

def initial_setup(pdf_path):
    """Executes the end-to-end ingestion pipeline: loads, cleans, chunks, and vectorizes the PDF."""
    chunks = load_and_clean_pdf(pdf_path)
    vectorstore = build_vectorstore(chunks)
    return vectorstore

Groq LLM

gpt-oss-120b
User query given to this LLM.
Get the response from the LLM for the user query.

Retriever

Sending the Query to the embedding.
[ TBD ]

Build the Prompt

[ TBD ]

Notes

We need some tool to learn the Model , those tools are called Ollama , lama.cpp , etc
If you want to run Python script , what you need ? Python right ? Like wise we need Ollama tool to run the Model.
Model is just like a file.
Zed( Editor ) is like Windows VS Code Editor
Langchain is used to create the Agent , its like a single Framework.
If its scanned PDF then reorganization model needs to be used then also need to use OCR.
White space regular expression.
Import re --> regular expression.
Unicode - utf8 --> Its a text format or Unicode format.
Langchain is like Broker , so it supports OpenAI , Claude , etc .
Langchain as many contacts so its a broker.
Even Chroma is included in the Langchain ECO-SYSTEM. Otherwise we need to install and write program for integration of Chroma.
Langchain is a wrapper.
Langchain satisfies the end user , you can buy everything here. That the idea of Langchain.
Langchain & Chroma DB is a open source.
Chroma DB in cloud is Paiza.
PineCone / PG Vector ---> [ TBD ]
SQL Query will not work in Vector DB.
What the retriever.invoke is doing ? [TBD]
The input of the PDF is SOFTWARE ENGINEERING.
Recursive means it tries to cut at nice places first: paragraphs, then lines, then sentences, and only cuts in the middle of a word as a last resort.

Error

Surrogates not allowed

Questions

RAG ----> Retrieval-Augmented Generation: find the relevant paragraphs first, then answer using only those.

Why RAG? ----> The model hasn't read your documents. RAG is cheaper and fresher than pasting everything or retraining.

Indexing vs querying ----> Indexing: prepare the document once. Querying: for each question, find chunks and answer.

Document loader ----> Reads a file into Documents, e.g. PyPDFLoader gives one Document per page with page_content and metadata.

Chunk ----> A piece of text (about 1,000 characters) small enough to search and send to the model.

chunk_overlap ----> Repeated characters between neighbouring chunks, so sentences cut at an edge are still whole in one chunk.

RecursiveCharacterTextSplitter ----> Cuts at nice places first (paragraphs, lines, sentences) and mid-word only as a last resort.

Embedding ----> A list of numbers for a text's meaning. Similar meaning = close together.

Vector database ----> Stores chunks with their embeddings and finds the closest ones to a question (Chroma, here).

Retriever and k ---> Finds the best chunks for a question. k = how many chunks to return.

RAG prompt ----> "Answer using ONLY the context. If it's not there, say you don't know." plus context and question.

Show sources ---> Each chunk has its page in metadata. Show it so readers can check the answer.