作者開發 Finder Lite，為 macOS 提供本機語義搜尋
I built the smallest version of local semantic search for macOS
Harish Kotra 開發 macOS 本機語義搜尋工具 Finder Lite，預設採用 EmbeddingGemma 2 的文字版。工具可索引文字、Markdown、程式碼及 PDF 文字；文字版為 270M 參數，Q8 下載量 295 MB，索引存於單一 SQLite 檔案。
Spotlight matches names and indexed words. That works until the thing you remember is an idea and not a filename. "the clause about pets", "the retry logic we wrote for 429s", "where we decided
against the annual plan". You know the file exists. You cannot name it.
A local embedding model solves that, and in October 2026 Google made it easy: EmbeddingGemma 2 is Apache 2.0, 740M parameters in full, and small enough to run on a laptop. The obvious build is the
full multimodal one. I went the other way and built the smallest useful version: text, Markdown, code and PDF text, a 295 MB download, one SQLite file, and about 4,300 lines of Swift.
It is called Finder Lite. Everything below is measured on an M2 mini with 8 GB of RAM, and every number is reproducible with a command in the repo.
The choice that defines the project
EmbeddingGemma 2 loads in pieces: a 270M text backbone, a 170M vision encoder, a 300M audio encoder. Full multimodal is 740M and, quantised, an 865 MB download. Text only is 270M and 295 MB in Q8.
That is a 2.9x difference in the only number a user sees before they have tried your app. On an 8 GB Mac it is also the difference between comfortable and tight. And the honest reading of the feature is that most of what people search for in their own files is text: notes, documents, source code, PDFs.
So: text only, and say so on the first screen instead of burying it in a README.
The stack, and why it is boring
- Swift, SwiftUI, one SwiftPM package. No Xcode project, no dependencies to resolve by hand.
-
llama.cpp, linked as the official prebuilt Apple
xcframeworkfrom a pinned release with a checksum.swift builddownloads it once. Metal is already enabled in that build. - SQLite for everything: folders, file state for incremental runs, chunk text, vectors.
- Accelerate for the query: brute-force cosine over every stored chunk.
No vector database. No second service. No Python. At 256 dimensions a chunk's vector is 1 KB, so 100,000 chunks is 100 MB of floats and a vDSP_dotpr loop answers a query in a few tens of milliseconds. An approximate index would be faster and much more code, and this is the version I wanted to be able to read end to end.
What it looks like inside
Two targets. The core library does all the work and imports no UI framework at all, which is why every part of this is drivable from a terminal. The app target is the views, the state machine and the
command line.
first run every search
┌──────────────────┐ ┌──────────────────────┐
│ download 295 MB │ │ type a sentence │
│ verify sha256 │ │ embed the query │
└────────┬─────────┘ │ 1 x 768 -> 256d │
│ └──────────┬───────────┘
┌────────▼─────────┐ ┌──────────────┐ │
│ pick folders │ │ llama.cpp │ │
│ (NSOpenPanel) │ │ Metal, 8 seqs│ │
└────────┬─────────┘ │ per encode │ │
│ └──────┬───────┘ │
┌────────▼─────────────────────▼────────┐ │
│ walk + extract + chunk + embed │ │
│ incremental: mtime + size decide │ │
└────────┬───────────────────────────────┘ │
│ │
┌────────▼───────────────────────────────┐ ┌──────────▼───────────┐
│ index.sqlite3 │ │ cosine over every │
│ folders / files / chunks(text + vec) │──▶│ chunk, vDSP, top-k │
└────────────────────────────────────────┘ └──────────┬───────────┘
│
┌───────────▼───────────┐
│ best chunk per file │
│ + small name boost │
└───────────────────────┘
Three pieces of code carry the whole thing.
Eight sequences per encode call. This is the difference between 13.3 chunks/s and the 9.8 chunks/s the same model manages behind llama-server. Each passage gets its own sequence id inside one batch:
var batch = llama_batch_init(Int32(flat.count), 0, Int32(texts.count))
defer { llama_batch_free(batch) }
var idx = 0
for (seq, len) in lengths.enumerated() {
for pos in 0..<len {
batch.token[idx] = flat[idx]
batch.pos[idx] = Int32(pos)
batch.seq_id[idx]![0] = Int32(seq) // which sequence this token belongs to
idx += 1
}
}
batch.n_tokens = Int32(flat.count)
let rc = isEncoder ? llama_encode(ctx, batch) : llama_decode(ctx, batch)
Note llama_batch_init and not llama_batch_get_one. The latter returns a batch that points at the
caller's token array, and Swift only guarantees that pointer for the duration of the call, so under
optimisation the batch dangles before the encode reads it. That cost me a release-only crash, and it
is why the self tests run against the release binary.
768 dimensions down to 256, then normalise. The model is Matryoshka trained, so the first 256
values are a usable embedding on their own:
var v = Array(UnsafeBufferPointer(start: ptr, count: nativeDim))
if dim < nativeDim { v = Array(v[0..<dim]) } // 768 -> 256
var norm: Float = 0
for x in v { norm += x * x }
norm = sqrt(norm)
if norm > 0 { for i in v.indices { v[i] /= norm } }
Every vector is unit length by the time it is stored, which is what makes the query a plain dot
product. The whole index is one contiguous [Float] buffer, so a query is one loop:
matrix.withUnsafeBufferPointer { m in
q.withUnsafeBufferPointer { qp in
guard let mb = m.baseAddress, let qb = qp.baseAddress else { return }
for i in 0..<n {
var dot: Float = 0
vDSP_dotpr(mb + i * dim, 1, qb, 1, &dot, vDSP_Length(dim))
scores[i] = dot
}
}
}
And the prompt format travels with the model, because using the wrong wrapper silently hurts
ranking rather than failing:
public static let embeddingGemma2 = PromptFormat(
documentTemplate: "title: {title} | text: {text}",
queryTemplate: "task: search result | query: {query}")
public static let nomic = PromptFormat(
documentTemplate: "search_document: {text}",
queryTemplate: "search_query: {query}")
The numbers
$ finder-lite bench ~/.hermes/skills --files 100
model: EmbeddingGemma 2 | 768d native -> 256d stored | mean pooled, non-causal | batch 8
files: 100, chunks: 627
embedding: 13.3 chunks/s, 2.12 files/s
projection: 1,000 files ~= 8m, 10,000 files ~= 1.3h
The same model served by llama-server on the same machine measured 9.8 chunks/s. The app is faster
than its own server path because it packs 8 sequences into a single encode call instead of letting
the server schedule them.
The uncomfortable number is the first pass. 10,000 files is about 1.3 hours. That is not a bug to
fix, it is the cost of reading and embedding every file once, and it shaped the UI more than anything
else: the app walks your folders and tells you what you are in for before you commit ("about 12,430
files, roughly 1h 40m on this Mac"), using the rate it measured on a previous run rather than a
guess. Indexing runs on a background queue, it can be paused and resumed, and search works against
whatever is already indexed. A first run that blocks the app for an hour is a first run nobody
completes.
What it can run on, and what it can run as
The obvious next question was "can I use a different model", and the honest answer needed care. Two
things are easy to get wrong. The first is that each of these models wants its text wrapped a
specific way: EmbeddingGemma 2 wants title: ... | text: ... and task: search result | query: ...,
Nomic wants search_document: and search_query:, BGE wants an instruction on the query only. Use
the wrong one and ranking gets measurably worse, silently. So the prompt format belongs to the model,
not to the app.
The second is that vectors from two different models are not comparable. Searching across a mixture
would return confident nonsense, so each model gets its own index file. Switching back to a model you
already indexed does not cost you another first pass. I checked that by indexing 40 files under
EmbeddingGemma 2, switching to BGE (which correctly showed an empty index), indexing that separately,
then switching back and finding the original 40 files still there.
EmbeddingGemma 2 stays the default. Nomic and BGE are in there so the choice is real, and both were
downloaded, hashed and checked against the same ranking test before being listed. You can also point
it at any GGUF you already have.
It also grew up as an app: a menu bar item that reports what is indexed and can pause or start a run,
a global shortcut that brings the window forward from anywhere, and a Settings window where the
model, the shortcut, the indexing rules and the startup behaviour all live. The shortcut uses Carbon's
RegisterEventHotKey rather than a global event monitor, because a global monitor for key events
requires Accessibility permission, and asking for that to open a search box is a bad trade.
Fifteen traps, all of them real
1. The encoder asserts on batch size. My first benchmark run died with
GGML_ASSERT(cparams.n_ubatch >= n_tokens && "encoder requires n_ubatch >= n_tokens"). For this
model any single call must fit inside the ubatch. The context, batch and ubatch are now all sized
from maxTokens x batchSeqs, and the tokenizer shrinks any input that still overflows.
2. The output width is not the hidden width. llama_model_n_embd returns 512. The usable
embedding is 768, from llama_model_n_embd_out. Read the wrong one and you get a truncated vector
with no error, no warning and a slightly worse index. I only caught it because the model loader logs
n_embd = 512 and n_embd_out = 768 next to each other. This is the quietest way to ruin a
retrieval system.
3. The model is not causal. It is mean pooled and non-causal, so it behaves like an encoder.
llama_decode silently routes to llama_encode for it, which means the naive code is accidentally
correct. That is worse than being wrong, because you cannot tell whether it is working. The
selftest command exists to answer that: it embeds the same passages two different ways and
requires them to agree to 1e-3.
4. Matryoshka truncation is free until it is not. 768 to 256 dimensions costs almost nothing and
cuts storage 3x. 128 dimensions is a real quality loss, especially for anything multimodal. 256 is
the default here and 128 is available but not recommended.
5. llama.cpp logs everything. It dumps every tensor on load, thousands of lines. A log callback
drops INFO and DEBUG, keeps warnings, and filters the one expected warning that fires once per encode
call. Without it, both the CLI and the app console are unreadable.
6. Metal compiles shaders on first use. The first ever run on a machine spent about 80 seconds
compiling the embedded Metal library. The second run took 0.32 seconds because the system cached it.
If you ship a Metal app and your first launch looks like a hang, this is why.
7. ggml asserts on the way out. The Metal device registry is a C++ static and it requires zero
references when the process tears down. Anything still holding a model at exit aborts in
ggml_metal_device_free, so the app can crash while quitting, long after the work is done. My first
guess was llama_backend_free and that was wrong: the fix is to release the model before
termination. I only found it because I wrote a headless mode that drives the app's own state machine
and returns an exit code.
8. A Pause button that lied. This one came back from a real user, which is the best kind of bug
report. Pressing Pause did nothing: indexing ran to completion while the status line said paused. The
cause is a one liner with teeth. The cancel flag was set through the same serial queue that the
indexing run occupies for its entire duration, so the write queued up behind the very work it was
supposed to interrupt. It landed the moment indexing finished, which is the one moment it is
useless. Anything that has to interrupt work must not share that work's queue. The flag now lives
behind a lock, and pausetest fails the build if it ever regresses. The UI changed too: Pause now
says "pausing after this file" and only claims "paused" once the indexer confirms it stopped and
committed.
9. Two spellings of the same path. FileManager.enumerator resolves symlinks, so a folder opened
as /var/... produces files stored as /private/var/.... Removing that folder then matched nothing,
because the prefix comparison never lined up, and its rows stayed in the index forever. Found by
checking the row count before and after a test that removes a folder. Every path now goes through
one canonical helper on the way in and on the way out.
10. A struct named after something SwiftUI already owns. I called the settings type Settings,
which is also a SwiftUI scene. Nothing failed in an obvious way. state.settings resolved to the type
rather than the property, and the fallout was about forty errors saying Bindings could not be
converted to Bool. Renaming it to AppSettings fixed every one of them at once. If a dozen unrelated
type errors appear together, suspect a name collision before you suspect your logic.
11. Two code paths, one set of rules. The app honoured the indexing settings and the command line
quietly used the defaults, so "skip hidden files" and the exclude list did nothing from a terminal.
Caught by indexing a fixture with a hidden file, a node_modules file and a 1.3 MB file and counting
what landed under two different settings. Both paths now build the same options from the same file.
12. NSApp is nil outside a GUI process. The headless smoke test crashed on the line that sets
the activation policy. Anything touching the application object is now guarded, which is also what
lets the same binary work as a CLI.
13. attributesOfItem reports a symlink's own size. A model symlinked from an external drive read
as a broken download, because the size check compared the length of the link against the expected
weight size. Symlinking a 300 MB model is a reasonable thing to do, so the path is resolved first.
14. An array literal that lied, but only in the release build. The strangest one. Six strings
assigned to an existing array variable inside an if branch produced an array whose count was 17
in the optimised build. The first six elements were correct, so the loop ran off the end into garbage
and the binary segfaulted without printing a single line. The same strings as a let were fine.
Building the array with append calls was fine. The debug build never reproduced it. That is the
second release-only miscompilation in this project, which is why the release binary runs the whole
self test suite and not just the debug one.
15. A crash on a machine you cannot test on. Built and tested on macOS 15.5, reported as crashing
on macOS 27.0.1: the window appeared, showing the first-run screen, and the process died. No crash
report the user could find and nothing printed. The fix was not a guess at the cause. The app now
keeps a launch log, one line per startup step, closed immediately so nothing sits in a buffer waiting
to be lost, with signal and uncaught exception handlers that append the step and the signal, and the
previous run's crash line is shown on the first screen. Verified by making it segfault on purpose:
the next run reported crash: died during crashtest: about to die on purpose (SIGSEGV).
Two details that cost time: a signal handler may not allocate, so the bytes it writes are built
before the crash and copied with write; and a closure that touches anything outside itself cannot
become the C function pointer that signal and NSSetUncaughtExceptionHandler require, so both
handlers are top-level functions.
The follow-up report was a frozen window rather than a crash: the app opens, the first screen appears,
and then the main thread blocks. A hang leaves no crash line at all, and my own trace command deleted
the log before reading it, which threw away the only evidence. Both are fixed: the log rotates to
launch-previous.log, a run that never finished starting up is reported by name, and a watchdog writes
stall: startup had not finished after 15s, still in <step> so even a force quit leaves a named step.
Verified by starting the app with no window server so it genuinely hung, then reading the next run's
report of it.
What is in the index
folders (id, path)
files (id, path, mtime, size, chunks, indexed_at)
chunks (id, file_id, ord, offset, text, vec)
files.mtime and files.size are the whole incremental story: unchanged files are skipped, changed
files are re-embedded, deleted files lose their chunks. On a large corpus the second run takes
seconds. The chunk text is stored next to the vector, which is what lets a result show the matching
passage instantly and opens the door to hybrid keyword ranking later.
How I know it works
Three checks ship with the repo, and none of them need the network.
$BIN selftest # the pipeline agrees with itself, and ranks meaning over keywords
$BIN smoke # the app's own state machine, including which screen it would show
$BIN bench ~/Documents # what this machine actually does
$BIN index <folder> && $BIN search "<a sentence you could never use as a filename>"
The third one is the only test that matters to a user. Search for a sentence that shares no words
with any filename in your corpus, and see whether the right file comes back. If it does, the whole
chain worked.
What I would do next
Hybrid ranking first (a BM25 or FTS5 pass beside the vectors, so invoice numbers and error codes match
exactly), then an FSEvents watcher so new files appear without pressing Index now, then DOCX and RTF,
which are both zipped XML and both common. After that: a score floor so weak matches can be hidden, and
index compaction, because deleting files removes their rows but the SQLite file does not shrink.
What I would not add is the multimodal encoders. That is a different app, and it already exists.
Code & more: https://www.dailybuild.xyz/project/286-finder-lite
來源：DEV Community · dev.to