Kikanai 用本地 Gemma 摘要語音訊息，讓用戶免於逐段收聽

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady.

In Brazil, nobody types. WhatsApp voice notes are how families, friends and half of the people you do business with talk, and every family has a Tia Cida: the aunt who forwards you seven minutes of audio to say there is a barbecue on Sunday and you should bring ice.

You can listen to it at 2× and still lose three and a half minutes. Or you can do what everyone secretly does:

So I built the lazy option properly. Kikanai (聞かない) is Japanese for "I won't listen".

What I Built

You drop a voice note on a page running on your own computer, and you read something like this instead of playing it (this one is the example on the banner):

Tia Cida
Churrasco no domingo, 13h.
Leva gelo. Não precisa responder.

☐ Levar gelo · domingo 13h
−7m 01s · enrolação 84% · no need to reply

Two lines, at most. The things you have to do, with when and where. Whether you actually need to answer, which is the real question behind every voice note. And two numbers: the time you did not spend listening, and the enrolação (rambling) index, the share of the audio's words that didn't make it into the summary.

Who gets to be lazier: anyone who receives long voice notes, which in Brazil is everyone. You can drop the file on the page, or just save it into a folder: Kikanai picks it up, summarizes it into your history, and moves it to done/. You never open the audio.

And nothing leaves your machine. ffmpeg converts the audio, whisper.cpp transcribes it, and Gemma 4 on Ollama summarizes it, all locally. Your aunt's health updates, your friend's breakup, your landlord's excuses: none of it goes to anyone's server.

Demo

Kikanai runs on your computer, never on a public server, so there is no live link. This is a real run on an 18-second test recording, shown at 2× speed:

The left side is where audio goes: the drop target is drawn like a forwarded voice note, and the whole window accepts the file. The stages light up as the server reports them, with a clock that doesn't lie. The right side is paper: the summary slides out from under the blue line, and every summary stays in the stack below.

The first time you open it, the page shows an example slip (clearly labeled exemplo) so you know what is coming. If something is missing, like the Whisper model or Ollama, it says so above the drop target, with the command that fixes it.

Code

mauricioandrade / kikanai

Voice notes you never have to play. Drop a long WhatsApp voice note, read two lines: whisper.cpp transcribes, Gemma 4 on Ollama summarizes, Java counts the time you saved. Java 25 + Quarkus, 100% offline.

MIT licensed. The README has the full setup (whisper.cpp build, model checksum, Ollama on Windows with Kikanai in WSL2) in English and Portuguese.

How I Built It

Two open models, each doing what it is good at

Gemma 4's small models can take audio directly, so my first idea was to skip Whisper. But they take about 30 seconds of it, and the voice notes worth summarizing are the long ones. So each model does its job: Whisper is a speech recognition specialist that handles long audio, and Gemma does the part that needs understanding, deciding what matters, what has to be done, and whether a question was asked.

Composing two specialized open models like this is the point of open weights. Nobody sells me this pipeline; I just plug the pieces together.

Gemma answers in a schema, not in prose

Asking a model to "answer in JSON" and hoping is how you end up parsing with regex. Ollama has structured outputs: you pass a JSON Schema in the format field, and generation is constrained to it. The schema is the contract:

{
  "type": "object",
  "properties": {
    "remetente": { "type": "string" },
    "resumo": { "type": "array", "items": { "type": "string" }, "minItems": 1, "maxItems": 2 },
    "acoes": { "type": "array", "items": { "type": "object",
      "properties": { "descricao": { "type": "string" }, "quando": { "type": "string" } },
      "required": ["descricao", "quando"] } },
    "precisaResponder": { "type": "boolean" }
  },
  "required": ["remetente", "resumo", "acoes", "precisaResponder"]
}

The keys are in Portuguese on purpose: the prompt is in Portuguese, and the model fills them more reliably that way.

The laziness numbers are arithmetic, not AI

If Gemma made up "84% rambling", the number would change on every run and mean nothing. So the model never sees the metrics. They are plain Java in the domain, reproducible and unit-tested:

// Words ÷ 200 wpm, rounded up to whole seconds, never below one second.
static Duration readingTimeOf(int words) {
    long seconds = Math.ceilDiv(words * 60L, READING_WORDS_PER_MINUTE);
    return Duration.ofSeconds(Math.max(1, seconds));
}

// 1 − summary/transcript: the share of words the listener didn't need. 0.84 means 84% filler.
static double ramblingIndexOf(int summaryWords, int transcriptWords) {
    if (transcriptWords == 0) {
        return 0;
    }
    double index = 1 - (double) summaryWords / transcriptWords;
    return Math.clamp(index, 0, 1);
}

The AI does what only it can do. The counting stays in code.

What Gemma got wrong, and how I fixed it

My test recording is me saying that it is a transcription test and that I can't remember the app's name. Gemma's first summary:

Este áudio é um teste de transcrição do aplicativo Ki-Kanae.
O remetente está verificando o nome do aplicativo.

"This audio is a test… The sender is checking the app's name." It described the voice note instead of summarizing it. A summary about the message is a summary of nothing, and in a two-line budget it is half the output.

Fix one, the prompt. Saying "don't describe the audio" helped a bit. What worked better was stating what I wanted instead of what I didn't: write the note the speaker would have sent instead of the audio. That is exactly the voice of "Leva gelo." I also added one worked example, deliberately unlike my test transcript, so a test could never pass just because the model copied it.

Fix two, a guard. At temperature 0.2 it still slipped once in a while. Retrying the same question at low temperature tends to get the same answer, so the parser rejects lines about "o áudio", "a mensagem", "o remetente" or "a pessoa", and the retry sends Gemma its own answer plus what was wrong with it:

} catch (InvalidSummaryException e) {
    // At low temperature the same question gets the same answer; the retry says what was wrong.
    if (content != null) {
        messages.add(new ChatRequest.Message("assistant", content));
    }
    messages.add(new ChatRequest.Message("user", e.correction()));
}

In the real run that followed, Gemma wrote "O remetente…" on the first attempt, got rejected, and came back with "Teste de transcrição do aplicativo Ki-Kanae." Clean.

The parser, every time Gemma writes "O remetente está…".

Being lazy takes some engineering

• The page shows real progress. The pipeline takes seconds to minutes, and a spinner on a long wait feels broken. The same POST /api/voice-notes streams NDJSON when the page asks for it: one line as each stage starts, then the result. curl still gets plain JSON.
• The folder waits for downloads. Browsers download into .crdownload and rename to .ogg when done, so files that aren't audio are ignored, and a file only counts once its size stops changing. You can save a voice note from WhatsApp Web straight into ~/kikanai/inbox.
• An assembly line. Each voice note runs on its own virtual thread, and each stage has a lane one note wide: whisper (CPU) transcribes the next note while Gemma (GPU) summarizes the previous one, and no two notes fight over the same stage.
• Everything checks itself. GET /api/health/deps reports ffmpeg, whisper-cli, the model, Ollama and the Gemma model, with the fix for whatever is missing. The page shows that list on first run.

Java 25 doing real work

The pipeline returns a sealed result, and the web layer deconstructs it in a switch the compiler checks for completeness:

return switch (summarizeVoiceNote.summarize(voiceNote(file, sender))) {
    case Done(var summary, var report) -> Response.ok(SummaryResponse.from(summary, report)).build();
    case Failed(var stage, var reason) -> ProblemResponse.of(ProblemResponse.UNPROCESSABLE_CONTENT,
            CANNOT_SUMMARIZE, reason, stage);
};

A ScopedValue carries the processing id into every log line of every stage, adapters included, without threading it through method signatures, and it can't leak into the next file the way a ThreadLocal on a reused thread can:

[94f6fd01] Converting WhatsApp Ptt 2026-10-19 at 19.21.37.ogg
[94f6fd01] Transcribing 18s of audio
[94f6fd01] Summarizing 37 words
[94f6fd01] Done in 13s, saved 12s of listening

The upload endpoint sits on a virtual thread (@RunOnVirtualThread) for the whole pipeline, which costs no platform thread while it waits for whisper and Gemma.

Reviewed by my other project

Shishō (師匠, "master") is my opinionated Java linter: it explains why something is wrong and whether the architecture fits the problem. It reviewed Kikanai after every phase.

Clean Architecture for a weekend tool sounds like overengineering, and without context Shishō would say exactly that. So the repo declares its context in shisho.yaml: one person, short life, and infrastructure that will change (whisper.cpp could become another ASR, Ollama another runtime, the JSON files a database). Its verdict: Clean Architecture is adequate; the declared context pays its cost.

It also caught a real gap: errors were handled endpoint by endpoint, so they now all go through one set of @ServerExceptionMapper methods as RFC 9457 problem+json. And it got one wrong: it raised that lesson on the Ollama REST client interface, before any server endpoint existed. A false positive I now get to fix in Shishō itself. Final grade: 1st dan.

Why Does Open Innovation Matter?

Because these are the most private recordings people have. Voice notes are family health news, money, fights, gossip, people's voices. Sending them to a cloud API to save three minutes is a bad trade, and most people wouldn't make it if they thought about it. With open weights, the audio, the transcript and the summary never leave the computer, and the page doesn't even load a font from outside.

Because I could compose the right models. A closed audio API would have sold me one model to do everything, priced per minute of audio, which is exactly the wrong unit for a tool whose whole point is long audio. Open models let me pick a speech specialist and a reasoning model and wire them together.

Because I could see why it failed, and fix it. Running Gemma myself, I could read every raw answer, constrain its output with a schema at the grammar level, and send it my own correction on retry. The "O remetente está…" fix came from reading what the model actually wrote, not from guessing at a black box.

Because it costs nothing per voice note. The busiest family group in the world costs the same electricity as a quiet one.

The Lazy Payoff

The first version lost to just listening. I ran four personal voice notes from one afternoon through it (they stay private, so only totals here), and processing took 97 s for 1m 49s of audio. For short notes, that is a bad deal, and the honest clock on the page showed it.

So I asked where the time went, and the answer was not in Java. It was whisper on the CPU and Gemma on the GPU; Java took milliseconds. Virtual threads make waiting free, not computing faster, so the fix had to be in how the tools were used:

The third row hides my favorite bug of the week: the first warm-up loaded Gemma with Ollama's default context, so the first real request, asking for 8192, made Ollama reload the entire model anyway. The fix was one argument.

After tuning, on my machine (WSL2 with 12 threads and 7 GB of RAM, Ollama on the Windows side with an RTX 2060), my 18-second test takes 7.0 s: 0.2 s of ffmpeg, 5.3 s of whisper and 1.5 s of Gemma. And the same four voice notes:

Dropped all at once they take 43 s: whisper is the bottleneck now, so the assembly line only hides Gemma's part. And on average, 68% of the words in those voice notes never made it into the summary. The longer the note, the better the deal: transcription grows with the audio, the reading doesn't.

Me, reading two lines instead.

Prize Categories

Best Use of Gemma. Gemma 4 E4B runs locally through Ollama and does the part that needs understanding: from a raw Portuguese transcript, it decides what matters, writes it as the two-line note the speaker would have sent, extracts the actions with when and where, and decides whether a reply is needed. Its output is constrained by a JSON Schema through Ollama's structured outputs, validated, and corrected with its own previous answer when it slips, while every number on the page stays deterministic Java.

Kikanai is week 3 of my Hacktoberfest run, after Janela and Kakeizu.