Heirloom：用 Gemma + Whisper 將親友零散的語音備忘錄整理成家庭食譜（100% 本機運作）

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

Every family has one person who cooks by feel. Ask [[NAME, e.g. my Seeya]] how much coconut goes into his pol sambol and the answer is "enough." The recipe isn't written anywhere. It lives in his hands.

So I built Heirloom. You hit record while they cook and talk, rambling, interrupting themselves, telling you who taught them. Heirloom turns that into a proper recipe page:

• Clean recipe: ingredients with amounts, ordered steps, tips, tags and allergens
• Their story, kept: the memories and "secrets" from the memo, not just the instructions
• Ask them next time: when the memo is vague ("a spoon of Maldive fish"), Heirloom does not guess. It writes a question to take back to them
• Their voice, kept: the original recording plays inside the recipe page
• Cooking mode (one big step at a time for messy hands), a cookbook shelf, and a print-to-PDF family book you can actually hand over

The design rule I held to: a recipe app that invents quantities is worse than no app. Uncertainty becomes a conversation with the person, which is the whole point.

Demo

Live: https://heirloom-saza.onrender.com

A quick honest note: the hosted link runs in demo mode. A free Render instance can't run an LLM, so it returns a canned sample so you can click through the whole experience. The real pipeline (local Whisper + local Gemma) runs on your own machine, and the screenshots below are from that local setup. The free instance also sleeps when idle, so the first load can take about a minute.

Code

Heirloom 🍲

Turn a loved one's rambling voice memo into a beautiful family recipe page, 100% on your own computer, with open-source AI.

Built for the Hacktoberfest 2026 Weekend Challenge: Build for a Friend (DEV). One real person, one real problem: a grandparent who cooks the best food in the family and has never written a single recipe down.

No account. No API key. No bill. The voice, the stories and the health details of the people you love never touch a server you don't control.

Table of contents

1. What it does
2. How it works
3. Tech stack
4. Quick start (local)
5. Configuration
6. Deploy on Render
7. API reference
8. Project structure
9. Design decisions
10. Why open source matters here
11. Troubleshooting
12. Roadmap
13. License

What it does

…

MIT licensed, with a full README (local setup for Windows/macOS/Linux and a Render guide).

How I Built It

The pipeline: voice memo → faster-whisper (local) → Gemma via Ollama (local) → structured recipe → cookbook

• Speech-to-text: faster-whisper (CTranslate2, int8, runs on CPU) with voice-activity detection to skip silence and kitchen noise.
• Understanding: Gemma 3 (open-weight) served by Ollama. I pass a JSON schema in Ollama's format field, so even a small 4B model returns valid, typed recipe JSON instead of free-form text I'd have to parse and pray over.
• A prompt that refuses to invent: the model may never add ingredients, amounts or times that weren't said. Anything vague goes into an unclear list, which the UI shows as "Ask them next time." A normalise() step guarantees the response shape on the way out.
• App: FastAPI backend, and a plain HTML/CSS/JS frontend with no framework, no build step and no CDN (system fonts), so it works fully offline. The recorder uses MediaRecorder with a live waveform, and a print stylesheet turns the cookbook into a PDF book.
• Everything is one env var away from changing: OLLAMA_MODEL, WHISPER_MODEL, OLLAMA_URL.

I built it pair-programming with Claude as an AI coding assistant, then ran, tested and adjusted it myself.

Why Does Open Innovation Matter?

This project would be a different (and worse) product on a closed API.

• Privacy is the feature. A recording of an elderly relative, plus the allergies and health details they mention while cooking, is exactly what shouldn't be uploaded to a server I don't control. With local Whisper and Gemma, nothing leaves the laptop. There's no account, no terms of service, no retention policy to read.
• Swappable brains. Whisper's accuracy on Sinhala and Tamil lags English, so a bigger Whisper size is one variable away. If a better multilingual model appears next month, I change one line and keep every recipe.
• It outlives pricing pages. A family archive should still work in 20 years. Local models cost nothing per memo and work in the kitchen with no signal.

The honest trade-off: running locally on CPU is slower than a hosted API, [[add your real timing for one memo, e.g. "about X seconds per 1-minute memo with whisper-small + gemma3:4b"]], and a 4B model needs the "never invent" guardrails to stay trustworthy. For an archive of voices I care about, I'll take slower and private every time.

Prize Categories

Best Use of Gemma: Gemma 3 runs fully locally through Ollama and does the core work: turning a messy spoken memo into a structured, honest recipe via schema-constrained output.

Best Use of Render: Heirloom's web app is deployed on Render as a free web service, using a render.yaml Blueprint and a /healthz health check, and that is the live demo linked above. To be upfront about scope: the hosted instance runs in demo mode (canned AI responses), because a free instance can't run the models. The real Whisper + Gemma pipeline runs locally. The README documents the paid path for running Ollama as a private service on Render, which I haven't deployed.