RecallAI 如何利用計算機視覺打造第二記憶

RecallAI: Building a Second Memory with Computer Vision at Cal Hacks 13.0

This is a submission for the MLH x DEV Writing Challenge.

What I Built

What if AI could remember the things we forget?

At Cal Hacks 13.0, my team and I built RecallAI, an AI-powered visual memory assistant designed to help people remember where they last saw their belongings and retrieve those memories through natural conversation.

We’ve all experienced that frustrating moment when we can’t find our keys, headphones, wallet, or something else important. Instead of searching everywhere, imagine simply asking:

“Hey RecallAI, where did I last see my AirPods?”

And getting a response like:

“Your AirPods were last seen on your desk 23 minutes ago.”

That’s the idea behind RecallAI.

How It Works

RecallAI combines computer vision, persistent storage, and an AI agent to turn everyday observations into searchable memories.

Our architecture uses:

• YOLO for real-time object detection.
• Visual embeddings to recognise and distinguish registered personal belongings.
• SQLite to store timestamped observations and associated visual evidence.
• Claude to interpret natural-language questions and retrieve relevant memories.
• FastAPI to connect the computer vision pipeline, database, and user interface.

Rather than simply identifying objects, RecallAI builds a history of when and where those objects were observed.

My Contribution: Temporal Memory

One of my main contributions was developing RecallAI’s Temporal Memory system.

Object detection answers the question, “What am I looking at?”

Temporal Memory goes further by asking, “When did I last see it, and what do I know about its history?”

The system uses timestamped visual observations to support capabilities such as:

1. Last-Seen Recall

Ask when an item was last observed and retrieve its most recent recorded sighting.

1. Historical Memory

Search previous observations to understand where an object appeared at a particular point in time.

1. Time-Aware Queries

Retrieve objects based on elapsed time, such as belongings that haven’t been observed recently.

1. Natural-Language Retrieval

Connect historical observation data with the AI agent so users can ask questions conversationally.

A particularly important engineering consideration was distinguishing what the system actually observed from what it could only infer.

For example, an object disappearing from the camera’s view doesn’t necessarily mean someone moved it or left it behind.

By grounding responses in recorded observations, we aimed to make RecallAI’s memory useful without overstating what the camera actually knew.

Beyond Objects: Remembering People

We also explored a broader vision for RecallAI: helping people maintain meaningful relationships.

Imagine registering a friend, with their consent, through a scan or manual entry.

RecallAI could remember when you last connected and gently remind you when it’s been a while.

For example:

“It’s been three weeks since you last caught up with Alex. Maybe it’s time to reach out!”

This expands RecallAI’s potential beyond finding belongings into helping people stay connected.

Our vision is simple: AI shouldn’t just help us remember things. It should help us remember what matters.

Demo

GitHub Repository: https://github.com/Slickstarlol/CalHacksObjectFinder

Demo Video: [Insert your video or Devpost demo link]

A typical RecallAI workflow looks like this:

1. Register a personal belonging.
2. Allow the camera to recognise it during normal use.
3. Store its observation history, including timestamps and snapshots.
4. Move the item outside the camera’s view.
5. Ask RecallAI when and where it was last observed.
6. Receive a time-aware response grounded in the recorded evidence.

The result is an interface that transforms camera observations into memories users can actually retrieve.

Partner Technologies

Our project brought together several technologies, each responsible for a different part of the memory pipeline.

Computer Vision: YOLO enabled object detection, while visual embeddings helped match observations against registered belongings.

Claude: Provided the conversational layer, allowing users to ask questions naturally instead of manually searching through recorded observations.

FastAPI and SQLite: Connected the system’s components and provided persistent storage for the historical memory layer.

MLH Partner Technology: [Add any official MLH partner technology your team actually integrated, including the API, SDK, implementation details, challenges, and what you learned. If none was used, state that clearly.]

One of the most valuable technical lessons was learning how to integrate different AI components into one coherent application. Detection, recognition, storage, retrieval, and natural-language interpretation all needed to work together.

Hackathon Experience

Building RecallAI at Cal Hacks 13.0, hosted in Berkeley, California, was an incredible experience.

What stood out to me was the atmosphere: being surrounded by ambitious builders, developers, and students who were genuinely excited about creating things.

Coming from the University of Birmingham in the UK, travelling to California for Cal Hacks was an opportunity to experience a completely different technology ecosystem.

My team and I had to make decisions quickly, divide engineering responsibilities, integrate our work, and turn an idea into something tangible within the hackathon’s limited timeframe.

We also learned that a compelling project doesn’t necessarily need to solve every problem imaginable. Sometimes the strongest approach is identifying one meaningful problem and building a focused solution around it.

For RecallAI, that problem was memory.

Working on Temporal Memory was particularly rewarding because it connected the technical infrastructure of our application with something users could immediately understand: being able to ask where something was and receive an answer based on actual observations.

Beyond the code, Cal Hacks reminded me why I enjoy software engineering in the first place.

It’s the process of taking an idea, collaborating with other people, encountering unexpected problems, and eventually seeing something you built come to life.

What’s Next?

RecallAI has potential applications beyond locating everyday belongings.

We want to explore richer historical search, more reliable context-aware reminders, and ways to help people maintain their personal connections.

Ultimately, we’re interested in one question:

What if technology could help us spend less time searching for things we’ve forgotten and more time focusing on the people and experiences that matter?

That’s the future we started exploring with RecallAI.

⸻

Built at Cal Hacks 13.0.

GitHub: https://github.com/Slickstarlol/CalHacksObjectFinder

CalHacks #MachineLearning #ComputerVision #AI