跳到正文
DEV Community· Souvik Das·· 2 小時前AI 評分39

FoodTruth 以開放權重視覺 AI 揭示包裝食品的誤導宣稱

FoodTruth: Exposing Packaged Food Deception with Open-Weight Vision AI

AI 導讀

開源工具 FoodTruth 透過食品包裝正反面影像擷取營養與成分資料，並審核誤導宣稱及法規風險。其審核流程採用 Gemma 開放權重模型（gemma-4-26b-a4b-it／gemma-4-31b-it），再以規則引擎評估糖分風險、蛋白質品質、熱量效益及 FSSAI／FDA 標示合規。程式碼以 MIT 授權開源，並提出每月 $4.99 或每年 $39 的消費者訂閲方案。

正文

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

Modern supermarket aisles are designed to mislead. Packaged food companies spend billions on deceptive front-of-pack claims like "No Added Sugar", "High Protein", and "100% Natural", while burying metabolic traps in fine print on the back panel:

  • Hidden Sugars: Products marked "No Added Sugar" routinely contain Maltodextrin (with a Glycemic Index of 110, spiking blood sugar faster than pure glucose), liquid glucose, or laxative doses of polyols like Maltitol.
  • The "Protein Sprinkle" Trick: Snack bars claim "Rich in Whey Protein" in bold front lettering, yet their ingredient lists place Whey Isolate behind maida (refined wheat flour), sugar, palm oil, and soy flour — merely sprinkling it in to satisfy legal claim thresholds.
  • Unlawful Disease Claims: Brands blatantly violate food safety regulations (such as FSSAI Section 24 and FDA guidelines) by claiming their snack or shake can "cure diabetes" or "control hypertension."

FoodTruth is an open-source, AI-powered food regulatory auditor and nutritional deception detector. By snapping or uploading dual packaging photos (Front Branding + Back Nutrition Table & Ingredients), FoodTruth runs an automated, multi-tiered audit that strips away corporate marketing and delivers the unvarnished metabolic truth.

       Front Branding                  Back Ingredients & Nutrition
  [ "15g Whey Protein" ]           [ Sugar, Palm Oil, Whey Isolate (2%) ]
             \                                      /
              \                                    /
               ▼                                  ▼
         ┌──────────────────────────────────────────────┐
         │          FoodTruth AI Audit Engine           │
         │   • Gemma Open Vision Schema Extraction      │
         │   • Sweetener Glycemic Index (GI) Matrix     │
         │   • DIAAS Protein Bioavailability & Dilution │
         │   • P:Cal Metabolic Efficiency Yield         │
         │   • FSSAI / FDA Regulatory Claims Audit      │
         └──────────────────────────────────────────────┘
                                │
                                ▼
         ┌──────────────────────────────────────────────┐
         │             Actionable Verdict               │
         │   "Protein Sprinkle Trick Detected"          │
         │   Optimized For: Quick pre-workout carbs     │
         │   Avoid If: Diabetic / Strict Calorie Deficit│
         └──────────────────────────────────────────────┘

Who Would Pay For It? (The Side Hustle Business Model)

FoodTruth is built with a direct commercial monetization pathway:

  1. B2C Micro-SaaS (Consumer Grocery Companion): A mobile web app / PWA subscription ($4.99/month or $39/year) targeting diabetics, keto athletes, parents, and health-conscious consumers who want a 2-second barcode/packaging camera scan while walking supermarket aisles.
  2. Pro Coach / Dietitian Pantry Audits ($19/mo per seat): Fitness trainers, dietitians, and functional medicine practitioners who audit clients' pantry items and generate branded, PDF-exportable nutritional truth reports.
  3. B2B Pre-Print Packaging Compliance API ($49/audit or $199/mo): Emerging D2C clean-label food brands and packaging design agencies need to pre-audit their label artwork against FSSAI and FDA advertising regulations before committing thousands of dollars to physical printing and risking regulatory notices or product recalls.

Demo

  • Live Repository: https://github.com/dasouvik122005/FoodTruth
  • Local Interactive Setup: Clone the repo, launch FastAPI (python main.py) and Vite (npm run dev), then test live packaging scans or choose from pre-loaded real-world audit payloads (Yoga Bar, The Whole Truth, and SuperYou Wafer).

User Interface Highlights

  • Bespoke Editorial Aesthetic: Designed with an editorial high-contrast light layout, split-screen desktop experience, and fluid mobile-first ergonomics.
  • In-App Camera Viewfinder: Integrated mobile camera capture with canvas-based shutter action for instant grocery shelf snapping.
  • Deterministic Metric Cards:
    • Metabolic Sugar Risk: High/Moderate/Low risk calculation based on real Glycemic Index (GI) database mapping.
    • Protein Quality Tier: Classifies proteins by DIAAS bioavailability (Tier 1 Whey Isolate down to Tier 3 Wheat Gluten/Flour fillers).
    • Calorie Yield (P:Cal): Calculates what percentage of total calories actually comes from bioavailable protein vs caloric padding.
    • FSSAI Violation Alerts: Real-time legal citations identifying specific regulatory breaches.

Code

The entire codebase is open-source under the MIT license at GitHub:

GitHub logo dasouvik122005 / FoodTruth

FoodTruth uses Gemma to analyze food packaging, extract nutritional data, and audit for compliance and health warnings.

FoodTruth

FoodTruth is an AI-powered application that analyzes food packaging images (front and back) to extract nutritional information and ingredients. It uses the Gemini API for precise data extraction and runs comprehensive audits against predefined rules to verify compliance and highlight potential health warnings.

🚀 Features

  • Automated Data Extraction: Upload images of food labels, and the system automatically extracts nutrition facts and ingredient lists using Google's Gemini models.
  • Rule-Based Auditing: Runs safety and compliance checks on the extracted data, identifying things like hidden sugars, allergens, or misleading claims.
  • Interactive UI: A modern React frontend built with Vite and Tailwind CSS to easily review results.
  • Demo Mode: Includes pre-configured demo samples for quick testing.

🛠️ Tech Stack

  • Frontend: React 19, TypeScript, Vite, Tailwind CSS v4
  • Backend: Python, FastAPI, Uvicorn
  • AI/ML: Google GenAI (Gemini)

📋 Prerequisites

  • Node.js (v18 or higher)
  • Python (v3.10 or higher)
  • Google…

Key Architectural Highlights

FoodTruth is architected into three clean decoupled layers:

  1. Frontend: React 19, TypeScript, Tailwind CSS v4, dynamic camera capture stream via navigator.mediaDevices.
  2. Backend: FastAPI (Python 3.10+), Pydantic v2 strict schema validation, asynchronous image streaming.
  3. AI Vision & Audit Pipeline: Google Gemma open-weight models (gemma-4-26b-a4b-it / gemma-4-31b-it) extracting structured entities into Pydantic models, fed into a deterministic regulatory engine.

1. The Pydantic Extraction Schema (backend/models.py) 

class ExtractionData(BaseModel):
    product_name: str
    serving_size: Optional[ServingSize] = Field(default_factory=ServingSize)
    front_claims: List[str]
    macros_per_serving: Optional[MacrosPerServing] = Field(default_factory=MacrosPerServing)
    ingredients_raw: List[str]
    parenthetical_sub_ingredients: List[ParentheticalSubIngredient]
    disease_or_medicinal_claims: List[str] = Field(default_factory=list)
    has_nutrition_table: bool
    has_ingredient_list: bool

2. The "Protein Sprinkle Trick" & Sweetener Detection Algorithm (backend/rules.py) 

# Detect if a brand claims 'Whey Protein' on front, but buries it behind cheap fillers
def audit_protein(data: ExtractionData) -> ProteinAudit:
    ingredients = [i.lower().strip() for i in data.ingredients_raw]

    first_tier1_idx = -1
    for idx, ing in enumerate(ingredients):
        for t1 in PROTEIN_TIERS["TIER_1"]["sources"]:
            if t1 in ing:
                if first_tier1_idx == -1:
                    first_tier1_idx = idx
                break

    front_claims_lower = [c.lower() for c in data.front_claims]
    claims_whey = any("whey" in c for c in front_claims_lower)

    # In food labeling, ingredients are listed by descending weight.
    # If Whey is claimed on front but appears past index 2, it is a sprinkle!
    sprinkle_trick = claims_whey and (first_tier1_idx > 2 or first_tier1_idx == -1)

    return ProteinAudit(
        declared_g=data.macros_per_serving.protein_g or 0.0,
        source_tier=top_tier,
        true_source=true_source,
        sprinkle_trick_detected=sprinkle_trick
    )

How I Built It

The Open-Source AI Stack

FoodTruth is built around Gemma open-weight models (gemma-4-26b-a4b-it and gemma-4-31b-it), Google's family of lightweight, state-of-the-art open models.

Why Open-Weight Vision Models?

Analyzing food packaging is notoriously difficult for classical OCR:

  • Packaged labels are curved, glossy, wrinkled, or photographed under poor grocery fluorescent lighting.
  • Nutrition facts exist in non-standard tabular grids, dual columns (per 100g vs per serving), and multi-language panels.
  • Ingredients are packed into dense, micro-font comma-separated lists with parenthetical sub-ingredients (e.g. "Compound Chocolate [Sugar, Hydrogenated Vegetable Fat, Cocoa Solids, Emulsifier (INS 322)]").

Gemma's multimodal architecture performs holistic scene understanding: it locates both panels across arbitrary image orientations, transcribes ingredient text verbatim, normalizes nutritional metrics to grams, and outputs strict JSON matching our Pydantic schema in a single pass.

Hybrid AI + Deterministic Rules Architecture

Instead of asking an LLM to hallucinate whether a product is "healthy" (which produces vague, non-reproducible answers), FoodTruth employs a neuro-symbolic pipeline:

  1. Perception Layer (Gemma): Performs vision OCR and extracts ground-truth product facts (exact ingredient sequence, front claim quotes, declared macro numbers).
  2. Audit Layer (Python Deterministic Engine): Evaluates the extracted entities against an immutable glycemic index database (SWEETENER_DB), bioavailability tables (PROTEIN_TIERS), and statutory regex patterns (DISEASE_CLAIM_PATTERNS).
  3. Verdict Generation: Produces explainable, reproducible scores that can stand up to regulatory scrutiny.

Why Does Open Innovation Matter?

1. Sovereignty Over Personal Health Telemetry

What you eat and the medical conditions you manage (Type 2 diabetes, insulin resistance, celiac disease, hypertension) represent some of the most sensitive personal data that exists. Closed, proprietary AI platforms log every prompt and image scan to enrich user ad-tracking profiles. Open-weight models like Gemma can run locally on edge hardware or private self-hosted servers, ensuring zero health telemetry leaks to third parties.

2. Impartiality and Truth Without Corporate Censorship

Closed foundation models are operated by massive cloud conglomerates with deep commercial ties to multinational food and beverage advertisers. Can consumers truly trust a closed-source model's nutritional advice when proprietary guardrails might quietly downplay the harms of ultra-processed food formulations?

Open innovation guarantees algorithmic transparency. Every prompt, every weight tensor, and every rule in FoodTruth is public, auditable by independent nutrition scientists, and immune to corporate lobbying.

3. Democratizing Preventive Healthcare in Emerging Markets

Countries like India, Mexico, and Brazil are facing catastrophic surges in lifestyle diseases and diabetes fueled by ultra-processed food marketing. Closed API token fees ($0.03–$0.05 per multimodal call) make consumer scanning tools economically unviable for everyday shoppers in emerging economies. Open-weight models empower developers to run batch inference or deploy quantised models on community servers at near-zero marginal cost.

My Agent Session

This project was developed end-to-end with Antigravity, Google's agentic coding assistant. The agent session encompassed:

  • Designing the dual-panel OCR prompt and Pydantic validation schema.
  • Implementing the glycemic risk and "protein sprinkle trick" heuristic engines.
  • Refactoring the frontend into a responsive, editorial interface with mobile camera viewfinder integration.
  • Resolving edge cases with non-standard nutritional tables and missing panels.

You can inspect the agent transcript through DevRelay:
{% agent_session a56d77e3-18a5-48b8-b5ba-c0c56fea78a7 %}

My First User

During development, I tested FoodTruth with a friend who is actively managing pre-diabetes and trying to maintain a high-protein calorie deficit for weight loss.

We scanned a popular supermarket "High Protein Wafer" claiming "10g Whey Protein" and "No Added Sugar".

The Audit Results Shocked Him:

  1. The Protein Lie: Whey protein concentrate was placed 5th in the ingredient list — behind wheat flour (maida), maltitol, soy protein isolate, and hydrogenated vegetable oil. The front claim of "Whey" was pure marketing dilution.
  2. The Sweetener Trap: The product contained 12g of Maltitol, which triggers gastrointestinal distress and has a glycemic index that still impacts blood glucose.
  3. The Illegal Claim: The label claimed the product was "Clinically proven to reverse diabetes" — a direct violation of FSSAI advertising standards.

His Reaction:

"I've literally been eating two of these a day thinking I was hitting my protein goals and protecting my blood sugar. Finding out that the whey is basically just sprinkled on top of maida and vegetable oil is infuriating. If this were an app on my phone, I'd easily pay $5 a month just to scan items in the grocery aisle before buying them."

That feedback confirmed the side hustle thesis: consumers don't want generic calorie counters; they want a ruthless advocate that exposes label deception.

Built with passion for food transparency and open-source innovation during Hacktoberfest 2026.

來源：DEV Community · dev.to

#開源/倉庫#多模態#Google