如何以 React Native、裝置端 TFLite 和 Grounded RAG 建構離線優先的作物病害偵測器

The Reality of Smallholder Agriculture in Central America

In mountainous agricultural regions across Costa Rica and Central America, smallholder producers of coffee (Coffea arabica), cacao (Theobroma cacao), and banana (Musa spp.) cultivate plots where cellular connectivity is intermittent or completely absent.

When foliar and pod pathogens strike—such as Coffee Leaf Rust (Hemileia vastatrix), Cacao Frosty Pod Rot (Moniliophthora roreri), or Banana Black Sigatoka (Pseudocercospora fijiensis)—early and accurate diagnosis is critical. A delay of several days or a mistaken treatment can destroy an entire seasonal harvest.

Cloud-based computer vision APIs fail in this environment: when standing in a steep coffee parcel miles away from the nearest cell tower, an app requiring a round-trip network request is unusable.

To address this challenge, we built HealthyCropping, an open-source, non-commercial, offline-first Android application.

GitHub Repository: https://github.com/keylorpineda/healthycropping

Architectural Principle: 100% Offline by Default

HealthyCropping was architected so that no core user action—capturing a photo, running deep learning inference, evaluating disease severity, or registering a plot sector—requires an internet connection.

Field Camera Capture: Guided framing tips tailored to leaf underside (coffee/banana) or pod surface (cacao).
On-Device Preprocessing: In-memory 224x224 RGB decoding and normalization.
Offline Inference: MobileNetV3 TFLite models running locally in under 50ms.
Instant Severity Triage: Results reported immediately on screen with actionable status (healthy, watch, act now).
Local Persistence: Data written to expo-sqlite using client-side UUIDs.
Optional Cloud Bridge: Asynchronous batch synchronization with FastAPI and Gemini RAG once connectivity is restored.

Challenge 1: Edge Inference Without Freezing the UI Thread

Running deep learning models on mid-range and low-end Android hardware presents severe memory and latency constraints. Loading a full TensorFlow runtime or converting uncompressed models causes dropped frames, out-of-memory crashes, and excessive battery drain.

We addressed this through three key architectural decisions:

Specialist Models per Plant Organ: Instead of an oversized multi-class model trying to classify all plants at once, we trained dedicated MobileNetV3-Large architectures focused on specific plant organs: Coffee Leaf, Cacao Pod, and Banana Leaf.
Quantization: Models are exported and quantized to float16 and INT8, reducing file sizes to below 10 MB per model.
Hardware Acceleration via Fast TFLite: We utilize react-native-fast-tflite, which binds directly to native C++ TFLite runtimes using GPU/NNAPI delegates where available. On test hardware, inference consistently completes in under 50 milliseconds.

Challenge 2: The Tensor Parity Trap

A common pitfall in edge machine learning is training-to-deployment preprocessing drift.

A model trained in Python with Keras might achieve 95% accuracy on test splits, but once deployed in a mobile JavaScript environment, slight differences in image decoding, resizing interpolation, or RGB normalization formulas degrade real-world accuracy drastically.

To prevent this drift, we created a shared fixture test suite between the Python ML repository (ml/) and the React Native client (app/):

A reference test image is passed through Python preprocessing (ml/src/preprocessing/).
The resulting 224x224 RGB float array is exported as an exact numerical fixture.
A Jest unit test in React Native runs the mobile decoding pipeline (app/src/ml/preprocessor.ts) and asserts bit-for-bit parity against the Python reference tensor.

If any change is made to normalization statistics or channel ordering, the automated CI pipeline fails before any model reaches the mobile bundle.

Challenge 3: Offline Data Integrity with SQLite

Farms, plots, and scans are stored locally using expo-sqlite.

Because users work offline for days, primary keys cannot rely on auto-incrementing database IDs generated by a remote server. We generate client-side UUIDv4 identifiers for all entities.

When connectivity is eventually restored, an idempotent synchronization engine reconciles local changes with our remote backend:

Unsynced records are batched into a single payload.
The backend processes records with upsert semantics, eliminating duplicate entries caused by network drops during sync.

Challenge 4: Preventing Dangerous AI Hallucinations

Generative AI in agriculture presents a serious risk: an unconstrained Large Language Model might invent chemical mixtures, incorrect dosages, or illegal fungicides that harm crops, contaminate soil, or void organic certifications.

HealthyCropping uses a Grounded Retrieval-Augmented Generation (RAG) pipeline powered by FastAPI and Gemini:

Curated Agronomic Corpus: We ingest verified technical manuals from national institutions (such as Costa Rica's ICAFE for coffee and CATIE for cacao).
Strict Source Attribution: The RAG prompt strictly confines Gemini to the retrieved manual passages. It is instructed to reject answering if verified guidance is missing.
Structured Recommendations: Guidance is categorized into Cultural Controls (e.g., pruning, shade regulation) and Approved Phytosanitary Measures, always citing the exact manual title and page number.

Current Status and Open Source Opportunities

HealthyCropping is actively in development across 9 structured phases:

Phases 1 to 5 (Completed): Model pipelines for coffee, cacao, and banana; float16/INT8 TFLite export; Expo React Native client shell; offline SQLite schema; and camera triage interface.
Phases 6 and 7 (In Progress): FastAPI cloud synchronization, Firebase token boundaries, Cloudinary image ingestion, and Gemini RAG advisory integration.

We welcome open-source contributions across all layers:

Mobile Engineers (React Native, TypeScript, Expo, SQLite): Sync engine optimizations, battery profiling, and UI testing.
Backend Engineers (Python 3.12, FastAPI, Firebase Admin): Batch sync endpoints, async processing pipelines, and API security.
ML Engineers (TensorFlow, TFLite): Quantization-aware training (QAT), dataset validation, and mobile inference benchmarks.
Agronomists & Researchers: Manual curation, retrieval chunk validation, and field trial feedback.

Explore the Code

GitHub Repository: https://github.com/keylorpineda/healthycropping
Architecture Overview: https://github.com/keylorpineda/healthycropping/blob/develop/docs/ARCHITECTURE.md
Implementation Roadmap: https://github.com/keylorpineda/healthycropping/blob/develop/docs/PLAN.md
Contributing Guidelines: https://github.com/keylorpineda/healthycropping/blob/develop/CONTRIBUTING.md

Feedback, issues, and discussions are welcome.