厭倦上課記筆記？我為此開發了 LazyNotes

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady.

What I Built

I have been lowkey falling behind my lectures. I start each semester with a burning motivation to pay attention in class. BUT writing notes is just too damn tiresome, especially when I also need to pay attention to what my professors are saying. Because let's be honest, once they go off on a tangent, I have no clue what the hell they just said.

AND for some reason, none of them ever record their lectures despite it being compulsory for them 🫩.

HOWEVER, for International Sloth Day, I decided to just make an app that takes my phone recordings and makes notes for me, so I don't have to do the hard work 😄.

And that's how LazyNotes came about.

The idea is simple: record a lecture using your phone, upload it, and get organized, topic-wise study notes with headings, bullet points, and explanations.

You can also provide the topics covered in class and an optional syllabus to help the model understand the context.

The goal is to spend less time frantically copying everything down and more time actually understanding what's being taught.

Demo

And here's what the app actually looks like!

The demo shows me uploading a real lecture recording, providing the topics covered, and getting organized study notes.

Note: The processing portion has been cut for the demo. Actual generation time depends on the recording length and hardware.

Code

LazyNotes is built around open-source technologies and open-weight AI models.

GitHub: https://github.com/Zemksh/lazynotes

The project includes the mobile application, backend processing pipeline, transcription, note generation, and validation and recovery systems.

How I Built It

I went with a mobile app and a Python backend.

The workflow is straightforward:

Record → Upload → Transcribe → Generate Notes → Study

Here's the stack:

But What About Noisy Classrooms?

One problem I had to consider was that classroom recordings aren't exactly studio-quality audio.

There's always something going on. Students talking in the background, chairs scraping against the floor, awkward silences, and sometimes a professor who seems determined to speak from the opposite end of the room.

So I didn't want the transcription pipeline to blindly treat every second of audio as speech.

I'm using faster-whisper with Silero Voice Activity Detection (VAD).

Silero VAD helps identify which parts of the recording contain speech and which parts are likely silence or non-speech. This allows Whisper to focus its transcription on detected speech regions instead of processing the entire recording indiscriminately.

However, VAD isn't the same as noise cancellation. If someone is talking loudly in the background, or the professor's voice is drowned out, transcription can still make mistakes.

That's where the second part of my approach comes in.

Whisper provides numerical diagnostics such as average token log probability and compression ratio. I use these with lightweight Python heuristics to flag potentially unreliable transcription segments.

For example, unusually low transcription scores or suspiciously repetitive text can indicate that a segment might need attention.

It's not a perfect system, and these metrics don't directly tell us whether every word is correct. But they provide a useful way to identify suspicious output without making another expensive language-model call.

I also wanted this reliability analysis to stay separate from the generated notes. Nobody wants their revision material cluttered with confidence percentages and technical warnings.

The Part That Took Longer Than Expected

Getting a model to generate notes wasn't particularly difficult.

Getting it to generate useful notes, reliably, in a reasonable amount of time was a different story.

My initial approach used Gemma 3:4B to generate notes from chunks of the transcript and then repeatedly combine those chunks using more model calls.

It worked in theory. In practice, it was painfully slow on my CPU.

I also ran into problems with incomplete JSON responses, validation failures, and repeated explanations across transcript chunks.

So I started simplifying the pipeline.

Instead of asking Gemma to repeatedly merge its own output, I moved consolidation into Python. I added chunk-level checkpoints so completed work could be reused after failures, and tightened validation to catch malformed output.

I also realized that the notes themselves needed work.

The initial results were too paragraph-heavy and repetitive. They read more like essays than something I'd actually use to revise before an exam.

I changed the generation format to focus on headings, concise bullet points, and important explanations.

That was a useful lesson: generating more text doesn't necessarily mean generating better notes.

Keeping the Notes Faithful

I also wanted to avoid blindly trusting everything the models produced.

The transcript is treated as the primary source of what was actually taught. The topics and syllabus are only there to help with terminology and context, not to introduce information that wasn't covered.

The generated notes are checked against structural requirements and valid transcript references using Pydantic and Python.

These checks help catch malformed outputs and invalid references, but they don't guarantee that every explanation is factually correct or that every important point was captured.

That's still an area I want to improve.

Why Does Open Innovation Matter?

The biggest reason I chose open-weight models was control.

I wanted to understand what was happening at each stage of the pipeline rather than sending a recording to an API and treating the response as a black box.

With Whisper and Gemma, I can experiment with model configurations, inspect performance, change the generation process, and run inference locally.

That also means the core AI processing can run without sending lecture recordings to a commercial AI inference API.

Of course, there's a trade-off.

Running Gemma 3:4B on my laptop's CPU is considerably slower than using a powerful cloud GPU. I've spent a surprising amount of time watching token-generation logs.

But being able to inspect and optimize the system has taught me a lot about inference performance, structured outputs, and building around the limitations of smaller models.

For me, that's one of the most valuable parts of building with open AI technologies.

The Lazy Payoff

I tested LazyNotes using actual classroom recordings, including lectures of approximately 10 and 50 minutes.

One of the biggest improvements came from removing unnecessary model calls and making the generated notes more useful for studying.

In a recent 10-minute lecture test, transcription took approximately one minute, while note generation took around four to five minutes in a successful run.

That's still slower than I'd like, but it's progress from the much longer runs and failures I encountered earlier.

More importantly, the output is moving closer to what I originally wanted: notes I can open later and actually revise from.

There's still plenty to improve, especially generation speed, formatting, and reliability.

But LazyNotes has already taught me something useful about building AI applications:

The hard part isn't always getting AI to do something. It's getting it to do exactly what you need, without making everything unnecessarily complicated.

And if that means I get to spend a little less time writing notes in class, I'll happily take it.

My Agent Session

I used a Codex along with Opencode to build and debug LazyNotes,
particularly the Gemma inference pipeline and
checkpoint recovery system.

[View my development session]

Prize Categories

Best Use of Gemma

I'm entering Best Use of Gemma because Gemma 3:4B is at the core of LazyNotes' note-generation pipeline.

Rather than using a hosted LLM API, I run Gemma locally through Ollama to convert classroom transcripts into structured study notes.

A significant part of the project involved making Gemma work reliably on ordinary CPU hardware: managing long transcripts, controlling output length, validating structured JSON, handling failures, and reducing unnecessary model calls.

The ability to run and experiment with an open-weight model is what made this project possible in the way I wanted to build it.

Overall Winner — Slow and Steady

I'm also submitting LazyNotes for consideration for the Overall Winner prize.

I think LazyNotes fits the spirit of the challenge pretty well. The whole idea came from wanting to do less of something tedious so I could focus on something more important: actually understanding my lectures.

What started as a simple recording-to-notes app turned into a much bigger engineering challenge than I expected. I spent a surprising amount of time figuring out how to make small, locally running AI models produce useful notes without taking forever or failing halfway through.

It's still a work in progress, but that's also what made building it interesting.

At the end of the day, LazyNotes is my attempt at making one small part of university life a little less exhausting.

Less writing. More listening.