跳到正文
DEV Community· Ai2th·· 6 天前AI 評分45

Miktam Bible 第 2 部分:在手機上運行的 AI 查經助手

Miktam Bible | Part 2 — An AI Study Assistant That Runs on the Phone

AI 導讀

Miktam Bible 的 AI 查經助手在手機上運行 SmolLM2,並以離線資料庫檢索到的經文作為回答依據。模型以 4-bit 量化並採用 GGUF 格式;每次提問會先搜尋聖經,再取 10 節最相關經文生成回答。助手不使用 API 或伺服器,手機切換至飛行模式後仍可使用;Miktam Bible 正在 Google Play 封閉測試。

正文

Ai2th

Most AI features in apps are a thin wrapper around a cloud API. Miktam Bible's study assistant is different: the language model runs on the phone, and every answer is grounded in verses pulled from the offline database.


🧠 The model

The assistant runs a small SmolLM2 model, quantized to 4 bits in GGUF format, through llama.rn, the React Native binding for llama.cpp. The settings are chosen for phones:

context = await initLlama({
  model: nativePath,
  n_ctx: 2048,        // Mobile-safe context window
  n_batch: 256,       // Mobile-safe batch size
  n_threads: 4,       // CPU threads for inference
  n_gpu_layers: 0,    // 0 = CPU only
  use_mlock: false,
  use_mmap: true,     // Memory-map the model file
});

CPU-only with a memory-mapped model file keeps it working across the wide range of Android hardware. If the model has not been downloaded yet, or fails to load, the assistant falls back to showing the most relevant verses for your question directly.


📚 Answers grounded in Scripture (RAG)

A small model on its own will happily make things up. So every question goes through a retrieval step first:

Your question
  └── 1. Search the Bible (FTS5, with synonym expansion and stop words removed)
        └── 2. Take the 10 most relevant verses
              └── 3. Build the prompt: system prompt + those verses + your question
                    └── 4. Generate the answer on the phone
                          └── 5. Turn verse references in the answer into tappable links

The model is told to answer from the verses it was given, and any reference in its answer (for example John 3:16) becomes a link back into the reader.


🔮 What's next: hybrid search

Keyword search finds verses that share words with your question, not verses that share its meaning. The pipeline is already shaped for hybrid search: FTS5 results plus vector similarity (sqlite-vec), merged with reciprocal rank fusion. The missing piece is a small on-device embedding model (MiniLM); until it lands, the assistant uses FTS5 alone.


🔒 Private by design

No API keys, no server, no logging of questions. You can put the phone in flight mode and the assistant still works.


Next, Part 3: reading plans and reminders that keep up with you.

📥 Get Miktam Bible

Miktam Bible is in Google Play closed testing:

  1. Join the AI2TH Google Group with the Google account on your phone. Members are testers for every AI2TH Play app.
  2. Opt in at play.google.com/apps/testing/miktam.bible.
  3. Install from Google Play.

Different Play account? Request access here.

來源:DEV Community · dev.to