由紙本報告到洞見：如何用 Pinecone、Ollama 和 PaddleOCR 建立私人醫療 RAG 系統

Let’s be honest: looking at a blood test result or a medical lab report feels like trying to read ancient hieroglyphics. You see numbers like Albumin: 45 g/L or HbA1c: 5.7%, and your first instinct is to panic-search WebMD. What if you could build a private, secure RAG (Retrieval-Augmented Generation) system that understands your historical data and answers your questions using local LLMs?

In this tutorial, we are building a Private Medical Document Archive. We'll use PaddleOCR to digitize messy lab reports, Pinecone as our high-performance vector database, and Ollama to run powerful models like Llama 3 locally. By leveraging a RAG system with LangChain, we can transform static pixels into a conversational health assistant that tracks trends over time.

The Architecture: How it Works 🏗️

The system follows a classic ETL (Extract, Transform, Load) pipeline modified for AI. We extract text from images, embed them into vectors, and retrieve context to feed our local LLM.

graph TD
    A[Lab Report Image/PDF] -->|PaddleOCR| B(Structured Text)
    B -->|Recursive Character Splitter| C[Document Chunks]
    C -->|Ollama Embeddings| D[(Pinecone Vector DB)]
    E[User Question: 'How is my glucose trending?'] -->|Embedding| F[Vector Search]
    D -.->|Relevant Context| G[Prompt Template]
    F --> G
    G -->|Llama 3 via Ollama| H[Final Answer + Trend Analysis]

Prerequisites 🛠️

To follow along, you'll need the following tech stack:

• PaddleOCR: For high-accuracy medical terminology extraction.
• Pinecone: To store and query our medical embeddings.
• Ollama: To run llama3 or mistral locally for privacy.
• LangChain: The glue connecting our OCR, Vector Store, and LLM.

Step 1: Extracting Text with PaddleOCR 🔍

Medical reports are often shared as PDFs or photos. Regular OCR often fails on complex table structures, but PaddleOCR handles them like a pro.

from paddleocr import PaddleOCR

# Initialize PaddleOCR (supports Chinese and English)
ocr = PaddleOCR(use_angle_cls=True, lang='en') 

def extract_text_from_image(img_path):
    result = ocr.ocr(img_path, cls=True)
    full_text = ""
    for idx in range(len(result)):
        res = result[idx]
        for line in res:
            # line[1][0] contains the detected text
            full_text += line[1][0] + " "
    return full_text

# Example usage
raw_medical_text = extract_text_from_image("my_lab_report.jpg")
print(f"Extracted: {raw_medical_text[:100]}...")

Step 2: Vectorizing Data with Pinecone & Ollama 🌲

Once we have the text, we need to turn it into math (vectors). We'll use Ollama for local embeddings to ensure your sensitive medical data never leaves your machine during the vectorization process.

from langchain_community.embeddings import OllamaEmbeddings
from langchain_pinecone import PineconeVectorStore
from pinecone import Pinecone

# Initialize Pinecone
pc = Pinecone(api_key="YOUR_PINECONE_API_KEY")
index_name = "medical-archive"

# Local embeddings using Ollama
embeddings = OllamaEmbeddings(model="llama3")

# Logic to upload documents
def store_in_pinecone(text_chunks):
    vectorstore = PineconeVectorStore.from_texts(
        text_chunks, 
        embeddings, 
        index_name=index_name
    )
    return vectorstore

Step 3: The RAG Logic & Trend Analysis 📈

The "magic" happens when we ask the system to compare different reports. By retrieving multiple chunks from different dates, the LLM can identify if a value is rising or falling.

from langchain.chains import RetrievalQA
from langchain_community.llms import Ollama

# Initialize Local LLM
llm = Ollama(model="llama3")

# Set up the RAG Chain
def ask_medical_assistant(query, vectorstore):
    qa_chain = RetrievalQA.from_chain_type(
        llm=llm,
        chain_type="stuff",
        retriever=vectorstore.as_retriever(search_kwargs={"k": 5})
    )

    template = f"""
    You are a professional medical data analyst. 
    Use the following retrieved context from lab reports to answer the user's question.
    If the data shows multiple dates, perform a trend analysis.

    Context: {{context}}
    Question: {query}
    Answer:
    """
    return qa_chain.run(query)

# Example Query
# response = ask_medical_assistant("Compare my Hemoglobin levels over the last 3 reports.", my_store)

The "Official" Way to Handle Medical AI 🥑

Building a local RAG is a fantastic start for personal use, but if you are looking for production-ready patterns—especially regarding HIPAA compliance, advanced chunking strategies for medical tables, and high-concurrency LLM orchestration—you should check out the deep-dive guides at WellAlly Tech Blog.

The team at WellAlly covers advanced topics like Hybrid Search (combining keyword and semantic search) which is crucial when searching for specific medical codes or drug names that embeddings might miss.

Why This Matters 🚀

Standard RAG systems often struggle with "temporal awareness"—knowing that a test from 2023 is different from one in 2024. By using metadata filtering in Pinecone (adding date to your metadata), you can specifically ask: "How did my LDL change between my last two visits?"

This setup gives you:

1. Privacy: Ollama keeps the "brain" on your laptop.
2. Accuracy: PaddleOCR minimizes transcription errors.
3. Scalability: Pinecone handles thousands of documents with millisecond latency.

Conclusion

Stop letting your health data sit in a dusty folder or a forgotten PDF. With Pinecone, Ollama, and LangChain, you’ve built a system that doesn't just store data—it understands it.

What's next? Try adding a Streamlit UI to upload your reports via a browser!

Disclaimer: This is a technical tutorial. Always consult with a qualified healthcare professional for medical advice and interpretation of lab results. 🩺

Found this helpful? Follow me for more "Learning in Public" AI tutorials, and don't forget to visit the WellAlly Blog for more advanced engineering patterns! 🚀✨