我有 5,000 張截圖，卻找不到想要的：我打造 SnapRecall，讓你按記憶搜尋截圖

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

SnapRecall is a screenshot search tool. It watches your screenshots folder, lets an AI model read every image in the background, and builds a private search index. Later, you type whatever you remember about the screenshot ("rahul payment 2500", "docker permission error", "wo flight ticket wala") and the matching screenshots show up.

How I got the idea

I take a lot of screenshots. A payment confirmation, a WhatsApp chat I might need as proof, an error message I want to Google later, a ticket, a recipe, a random detail from a group chat. I never organize them, because the whole point of a screenshot is that it takes one second.

The cost shows up later. My screenshots folder has thousands of files, all named like this:

screenshot_2026-10-24-14-55-16.png
screenshot_2026-10-24-14-55-41.png
screenshot_2026-10-24-15-02-07.png

When I need one specific image, those names tell me nothing. I end up scrolling through hundreds of nearly identical thumbnails, squinting at each one, and sometimes giving up. It is not a big problem, but it happens often, and it is exactly the kind of tedious work that a computer should do instead of me.

When I read the "slow and steady" theme, this was the first thing I thought of. It is a task that should happen quietly in the background, with no effort from me, while I do something else.

Two ideas, one choice

I had two possible solutions:

Rename on save. The moment a screenshot is taken, an AI reads it, summarizes it, and renames the file with the important words. A WhatsApp chat about rent becomes something like whatsapp_rahul_rent_payment.png.
Search by memory. Leave the files alone. When I cannot find a screenshot, I type the words I remember from inside it, and the app shows me the matches.

Renaming looked simple, but when I thought it through, it had problems. A filename is one guess about what I will remember later, and the AI's guess is often not the thing I remember. A screenshot of a chat could be filed under the contact's name, the topic, or the amount, and I might search by any of them. Renaming files can also break references in other apps or sync tools, and after a few thousand files, long auto-generated names start to look the same again.

Search works with whatever I happen to remember, and it does not touch my files. So I chose search as the core, and I treat smart renaming as a possible add-on for later (with a preview and an undo, not silent changes).

Who it is for

Anyone with a messy screenshots folder: students who screenshot notes, developers who save error messages, and people who screenshot payments, chats and tickets as proof. I also built it with Hindi and Hinglish in mind, because that is how I actually think and type. A lot of the text in my screenshots is in Hindi or mixed, and I did not want a tool that only works for clean English.

Features

Background indexing: new screenshots are picked up automatically soon after they are saved
A slow, throttled backfill of your old screenshots, so a big backlog never needs your attention
A vision model reads each screenshot and produces a short summary, tags, an app type (chat, payment, code, error, ticket...) and the text it can see
Hybrid search: keyword matching for exact names, amounts and error codes, plus semantic search for vague memories
English, Hindi and Hinglish queries
A simple React interface with a thumbnail grid, a larger preview, and live indexing progress with pause and resume

Demo

Code

GitHub repo

How I Built It

The stack

Backend: Python and FastAPI
Frontend: React (Vite, TypeScript)
Storage: SQLite for everything, with FTS5 for keyword search and sqlite-vec for vectors
Folder watching: watchdog
Models: Gemma for reading screenshots, plus an embedding model for semantic search

I chose SQLite on purpose. For a personal tool, one file with no server to run is a big advantage, and it keeps the whole thing light on a laptop.

The pipeline

Screenshots folder
      |
      v
Watcher + folder scanner  ->  job queue (SQLite)
                                   |
                  one image at a time, with a pause
                                   v
         downscale image -> Gemma: summary, tags, app type, visible text
                                   |
                          embed the combined text
                                   v
        SQLite: metadata + keyword index + vector index
                                   ^
                                   |
   React UI -> FastAPI -> hybrid search (keyword + semantic, merged)

For each screenshot, the worker downscales the image (so it does not send more pixels than needed), asks Gemma for a short summary, a few tags, the type of screenshot, and the important text it can read, then turns that combined text into an embedding. Everything is stored in SQLite next to the original file path.

Why the search is hybrid

I tried to think about how I actually look for screenshots, and I realized there are two very different cases.

Sometimes I remember something exact: a name, an amount like "2500", an error code. Keyword search is best for this, because semantic search can blur exact tokens together.

Other times my memory is vague: "that screenshot of the bus ticket" or "the chat where we decided the date". Here, keyword search fails if the exact words do not match, but semantic search can still find it because it matches meaning.

So SnapRecall runs both searches and merges the two ranked lists with Reciprocal Rank Fusion, which rewards results that rank well in either list without needing to compare their scores directly. It is a simple technique, and it covers both ways I search.

Slow and steady, on purpose

This is where the theme became a design rule instead of a slogan. Reading thousands of images with a model is heavy work. If a tool tries to do it all at once, it either freezes your computer or burns through your API limits and fails halfway.

So the indexer is built to be patient:

It processes one image at a time, with a configurable pause between images
It backs off and retries when the API rate-limits it, instead of treating that as a failure
It can pause itself when a daily request budget is reached, and it can be paused and resumed from the interface
If the app is closed or crashes, it picks up where it stopped when it starts again

You can keep working while it runs. It is the opposite of a tool that demands your attention.

Local first, then hosted, and why I changed

My original plan was fully local: Gemma through Ollama, Tesseract for OCR, everything on my own machine. Then I looked at my laptop. It is old and weak, and running a vision model on it would be painfully slow, if it ran at all.

So for this submission I changed course and used Gemma 4 through the Gemini API. The backend does not become heavy this way: it only watches a folder, stores data in SQLite and serves a few endpoints, so my laptop handles it fine, and the heavy model work happens remotely. I also turned off the OCR step in hosted mode, because Gemma already returns the text it sees.

I built the model layer behind a provider switch, so the same pipeline can point to a hosted API or to a local Ollama server. That was partly a practical decision and partly a way to keep the project honest about the open vs hosted tradeoff, which I explain below. One thing to be clear about: in hosted mode, the embeddings come from a hosted embedding model that is not open-source, so that part of the stack is not open.

I also used an AI assistant to help plan the architecture and write parts of the code. I am still testing how well it all works together, which brings me to the status at the end of this post.

Why Does Open Innovation Matter?

The prompt asks some direct questions, so I will answer them directly, including the places where my answer is "not yet" or "not really".

Does running your own model mean it works without internet?
In principle, yes, and that is one of the main reasons I picked an open model family. In the mode I used for this submission, no: I ran Gemma through a hosted API, so it needs an internet connection. The project is designed so that a local Ollama setup should work offline once the models are downloaded, but I have not been able to test that on my own hardware, so I will not claim it works.

Does it keep personal messages and files on your own machine?
This is the point that matters most for this particular project. Screenshots are one of the most private things on a phone or laptop: OTPs, bank messages, personal chats, ID photos. Uploading thousands of them to a third party is a real cost, and many people would rightly refuse.

In hosted mode, my screenshots do go to the model provider, so please do not treat that mode as private. What open weights give me is a private option. The same model family can run on a machine the user controls, and a user who cares about privacy can choose that path. With a closed vision API, there would be no such option: the only way to use it is to send the images out.

Does it let you set it up once and never pay again?
With a local model, that is the idea: after the setup, indexing 10,000 screenshots costs only electricity. With the hosted route I used, I am on a free tier with limits, and those limits shaped the design directly. The worker is throttled, it respects rate limits, and it has a daily request budget, because indexing a big backlog through a hosted API would otherwise hit those limits fast. A closed API charged per image would add a cost on every re-index, which matters a lot for a tool whose job is to process thousands of images.

Where did the open approach work better than a closed one?
The honest answer is flexibility, not raw quality. Because Gemma's weights are open, the same model was available to me as a hosted API when my laptop could not run it, and it is available to someone with a stronger machine to run privately. I could pick the deployment that matched my hardware, and I can change that choice later without rewriting the project.

Prize Categories

Best Use of Gemma: Gemma is the model that actually reads the screenshots. Its summary, tags, app type and visible text are what the whole search index is built from, so without it the project does not work. I used Gemma 4 through the Gemini API.

Where This Stands

This is an idea I took from a real annoyance to a working design in a short time, and it still needs testing.

Thank you for taking the time to read this.