Fridge to List：用 Gemma 3 + Ollama 在本機將雪櫃照片轉成購物清單

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

Fridge to List turns photos of your fridge into a grocery list, so you don't have to stand there with the door open writing things down.

You upload one to four photos (shelves, door, pantry). A local vision model looks at them and reports only what it can actually see, with a confidence level for each item and a separate "could not tell" list for anything blurry. Then the app compares that against your own list of usual staples and builds an editable checklist you can copy or download as a text file.

The lazy part: you take photos and walk away. The effort it saves is the standing, the looking, and the writing.

Demo

Video walkthrough: https://youtu.be/6Ng662R7paE

I live in a hostel and don't have a fridge there, so the demo uses photos of my home fridge, which I received on WhatsApp.

Code

Public repo, MIT licensed: https://github.com/anshraj3060/fridge-to-list

How I Built It

The core is Gemma 3 (4B), an open-weight vision model, served by Ollama on my own MacBook Air. The app is a small Node.js server plus a plain browser UI. Photos are resized in the browser, sent to the local server, and passed to Ollama. The model has to answer in strict JSON: items it can see, a rough quantity, a confidence level, and a "could not tell" list.

I built the app with an AI coding agent (Arena) from a single prompt; I did not write the code by hand. What I did myself was set it up and test it: installed Ollama, pulled gemma3:4b, merged the agent's branch into main, ran the app, and tried it on real photos.

Design choices that matter:

• No fake output. If Ollama isn't running or the model isn't installed, the app shows an error with the setup commands instead of sample data.
• The photo can't show what is missing. So the list is an inference: your usual staples, minus what the model saw. The app says this on screen ("Missing is a guess, not a fact").
• Show your work. A "What the model actually saw" panel lists every item with its confidence, so you can verify and fix the list.
• Everything is saved in the browser only. Staples live in localStorage; photos are processed in memory.

Why Does Open Innovation Matter?

• Privacy. A fridge photo can show more about a household than people expect. With a local open-weight model, the photos go from the browser to a server on my own laptop and nowhere else.
• Cost. There is no per-photo API bill. Once the model is downloaded, running it again and again costs nothing but electricity, which is what makes a "lazy, run it every week" tool realistic.
• Offline. After the one-time model download, generation doesn't need the internet.
• Control. The model is a setting. I can point the app at a different Ollama model with one environment variable.

Where it was weaker than a closed model would likely be: a 4B model is small. My home-fridge photos were dim and blurry, and the model reported only a handful of things (containers, bowls, bottles, a cabbage, yogurt, tomato sauce) and put an unclear jar in "could not tell" rather than guessing. That is the honest behaviour I wanted, but it also means the list needs a human glance. Gemma is open-weight under its own license terms, not OSI-approved open source, and running it still costs hardware and electricity.

The Lazy Payoff

On my MacBook Air, the app reported 7.5 seconds to write the list from two photos (Gemma 3 4B, locally).

Doing the same by hand, from the same photos, took me [X minutes — to be filled in].

I only tested on photos of one fridge, so treat this as one data point, not a benchmark.

My First User

Nobody else has used it yet, so there is no first-user story to tell.

Prize Categories

Best Use of Gemma: the whole project runs on Gemma 3 vision through Ollama.