ContaCal 如何打造能辨識巴西菜式照片的 AI 熱量計算器

Most calorie apps are built around a search box. You type "rice", scroll through 40 entries with different numbers, pick one, guess the grams, and repeat for every item on the plate. In my experience that is the moment people give up. It isn't the diet. It's the data entry.

I'm Lucas Pinheiro. I've worked on digital products, SEO, ads and AI since I founded LucasPro in 2016. In 2025 I built ZapGastos, and in 2026 I started ContaCal, a calorie counter where you log a meal by sending a photo. I'm not a nutritionist, and this post isn't nutrition advice. It's about the engineering problem: turning a phone photo of a very Brazilian plate into a useful calorie and macro estimate.

The problem with "global" food recognition

A typical lunch in Brazil is rice, beans, a protein, salad, maybe farofa, all on one plate and often touching. Add regional dishes (baião de dois, feijão tropeiro, tapioca, açaí bowls with toppings), the marmita (a packed lunchbox), and comida por quilo (buffets where you pay by weight), and you get a lot of cases that generic food datasets cover badly.

There are two separate failure modes:

1. Recognition: the model sees "brown stuff with grains" and calls it chili instead of feijão, or misses the farofa sprinkled over everything.
2. Nutrition lookup: even when the item is named correctly, the reference values come from a database built around another country's recipes and serving sizes.

Fixing only the first doesn't help much if the second is still wrong.

The pipeline, at a high level

I won't go into vendor specifics, but the flow has four stages and each one can fail in its own way:

photo / text / voice
   -> 1. item recognition   (what is on the plate?)
   -> 2. portion estimation (how much of each?)
   -> 3. nutrition mapping  (match each item to a food in a Brazilian database)
   -> 4. aggregation        (calories, protein, carbs, fat, meal Health Score)

1. Item recognition. A vision-capable AI model gets the image and returns a structured list of items rather than prose. Asking for structure (item name, a rough preparation like "fried" or "grilled", and a portion guess) makes the later stages much easier to validate. Prose reads nicely, but code can't check it.

2. Portion estimation. This is the hardest part, and I'd rather say so than pretend it's solved. A single photo has no reliable scale. A plate, a fork and the lunchbox walls give some hints, but a deep pot of beans and a shallow one can look the same from above. We treat portions as estimates and make them easy to edit after logging.

3. Nutrition mapping. We don't let the model invent calorie numbers. The recognized items get matched to entries in a food database built for Brazilian meals (600+ foods today, including regional dishes, lunchbox staples and restaurant food), and the numbers come from those entries. Users can also look up a food directly before logging. That split gives a clear line between two kinds of error, which makes debugging much easier.

4. Aggregation. Per-item values get summed into calories, protein, carbs and fat for the meal, plus a simple Health Score, and the meal goes into the user's day. From the user's side the whole round trip takes about three seconds.

Text and voice use the same pipeline

Photos aren't always practical. Sometimes the food is already eaten, or you're on a bus. So the same flow accepts a typed description ("1 pão de queijo and a latte") or a voice note. Only stage 1 changes: we parse language instead of pixels, and stages 2 to 4 stay the same. One normalized "list of items with portions" format feeds everything downstream.

Meeting users where they already are: WhatsApp

In Brazil, WhatsApp is the default interface for almost everything. Asking people to open yet another app at lunch adds friction, so ContaCal also works through an official WhatsApp number. You send a photo, text or audio, get calories and macros back in the chat, and the entry syncs to the app dashboard.

On the engineering side this means the logging pipeline can't be tied to any one client. The app and the WhatsApp channel are both thin front ends over the same backend flow, and there's one source of truth for the user's day. When we fix something in recognition or in the database, both channels get it at once.

Things that were harder than expected

• Mixed plates. Rice and beans touching, sauces covering the protein, farofa on top. Splitting a single "blob" into separate items matters more than spotting exotic dishes.
• Same name, different food. Regional names vary, and the same dish changes a lot from one state to the next. The database has to carry those variations instead of one "average" entry.
• Restaurant and buffet food. Cooking oil and preparation are mostly invisible in a photo. That's a real source of error, and we say so instead of hiding it.
• Trust. A confident number that's wrong is worse than a rough number the user can see and adjust. Making the estimate editable was a product decision as much as a technical one.

The deterministic part: calculators

Not everything needs AI. Daily calorie targets, BMR, TDEE, macros and similar numbers come from published formulas. The site's calorie calculator, for example, uses Mifflin-St Jeor. We keep those as plain, deterministic calculators that anyone can use without signing up, and the app uses the same logic for goals. If you want to look at the math, the ContaCal calculators are open on the web.

What I'd tell anyone building something similar

• Keep the model on recognition and the database on numbers.
• Ask the model for structured output you can validate.
• Treat portion size as uncertain and design the UI around fixing it.
• Build the pipeline once, and plug in channels (app, chat, voice) as thin clients.
• Test with local food, not stock photos. Your users' plates are your real test set.

If you want to try it

ContaCal is a subscription app with a 7-day money-back guarantee, available on iOS, Android, the web and WhatsApp. If you want to see the photo-to-macros flow on a real plate, it's at contacal.com. I'm happy to answer questions about the architecture in the comments.

Note: ContaCal's numbers are estimates meant to help with logging and organization. They don't replace guidance from a doctor or a registered dietitian/nutritionist.