Siddhant Kumar
Project 031 · Agriculture

Crop Disease Camera.

A field camera node that photographs crop leaves, runs a trained disease classifier on-device, and flags early signs of infection — with a clear account of what the model can and cannot recognise.

Advanced 20–30 hours 38 min read CameraVisionFarming
Jump to source Bill of materials
Crop Disease Camera — reference build illustration
Difficulty
Advanced
Build time
20–30 hours
Indicative cost
₹3,200 – ₹6,500 depending on compute choice
Platform
ESP32-S3 DevKitC-1
Category
Agriculture
Last updated
28 July 2026
Contents — 27 sections

Project Overview

A field camera node that photographs crop leaves, runs a trained disease classifier on-device, and flags early signs of infection — with a clear account of what the model can and cannot recognise.

Plant disease, caught early, is often manageable — a targeted fungicide, removing affected plants, adjusting conditions. Caught late, it can take a field. But early detection requires close, frequent inspection of leaves that a farmer covering acres cannot do, and the visual signs are subtle and easy to miss until they are not. A camera node that automatically photographs leaves and runs a disease classifier on the images offers a way to inspect continuously and flag problems while they are still small.

This is a genuine and active application of edge machine learning, and it is also one where honesty about the limits is essential. A model trained to recognise a specific set of diseases on a specific set of crops does that well; shown a disease it was never trained on, or a crop it never saw, or a nutrient deficiency that looks like a disease, it will confidently produce a wrong answer. The model recognises patterns it has learned, not "disease" in the abstract. This documentation makes the scope of the model — which crops, which diseases — central, and frames the output as a flag for confirmation, not a diagnosis.

The build runs a trained convolutional network on an ESP32-S3 (with its vector instructions and PSRAM for on-device inference) or, for larger models, streams to a Raspberry Pi. It captures a leaf image, runs the classifier, and reports the most likely class with its confidence — flagging low-confidence or disease-positive results for a human to confirm. The famous PlantVillage dataset (54,000 labelled leaf images across many crops and diseases) is the standard training set, and its strengths and well-known weaknesses (lab conditions, single leaves on plain backgrounds) are discussed honestly, because a model trained on it can perform far worse in a real field.

The result is a node that continuously watches a crop and surfaces likely disease early, understood correctly as a screening tool whose job is to say "look at this, it might be diseased" to a person who can then confirm and act.

A USB webcam
A camera module. Sharp, well-lit, leaf-scale images are the prerequisite — disease signs are fine details a blurry image loses. Photograph sourced from Wikimedia Commons — Webcam.jpg. Reused under the licence stated on that page; please check it before republishing.

What this project does

  • Captures leaf images automatically on a schedule or trigger.
  • Runs a trained crop-disease classifier on-device (ESP32-S3) or on a Raspberry Pi.
  • Reports the most likely disease class and confidence for a defined set of crops and diseases.
  • Flags disease-positive and low-confidence results for human confirmation.
  • Handles the model's scope honestly — reports "unknown" outside its training.
  • Logs detections with images for review and for retraining.
  • Reports over the network for field-scale coverage.

Real-World Applications

SettingHow it is used
Early disease detectionFlagging likely infection while it is still small and manageable.
Greenhouse crop monitoringControlled conditions where the model performs closest to its training.
High-value cropsWhere the cost of a missed disease justifies dense camera coverage.
Scouting assistanceDirecting a scout's attention to plants the model flags.
Learning edge AIA complete train-quantise-deploy pipeline for a real vision task.
Research and demonstrationA platform for field-vision disease-detection studies.

Deployment contexts where a build of this kind earns its keep.

Features & Capabilities

  • On-device inference with a quantised CNN on the ESP32-S3, or a larger model on a Pi.
  • Defined, honest scope — specific crops and diseases, "unknown" outside them.
  • Confidence-gated flagging so uncertain results go to a human, not acted on automatically.
  • PlantVillage-trained baseline with the real-field performance gap discussed openly.
  • Field-image robustness techniques (augmentation, background handling) to narrow that gap.
  • Detection logging with images for confirmation and for collecting field data to retrain.
  • Scheduled or triggered capture for continuous or event-based inspection.
  • Network reporting for field-scale coverage.

Difficulty, Time & Required Skills

AttributeValue
Difficulty levelAdvanced
Estimated completion time20–30 hours
Indicative build cost₹3,200 – ₹6,500 depending on compute choice
Primary disciplineAgriculture
Reference platformESP32-S3 DevKitC-1

Skills you should have (or will pick up)

  • Python and deep learning (CNN training, transfer learning)
  • Model quantisation and edge deployment (TFLite Micro / ONNX)
  • Camera interfacing and image capture
  • Understanding of dataset bias and real-world generalisation
  • Honest evaluation and scope definition

Bill of Materials

Every part below is commonly available from Indian and international hobby-electronics suppliers. Prices are indicative 2026 retail figures in Indian rupees and will drift — treat them as a budgeting guide, not a quotation.

ComponentKey specificationQtyApprox. cost
ESP32-S3 DevKitC-1
The vector extensions roughly triple TinyML inference speed over the original ESP32.
Dual-core Xtensa LX7 @ 240 MHz, 512 KB SRAM + 8 MB PSRAM, vector instructions for ML, Wi-Fi + BLE 51₹900
ESP32-CAM (AI-Thinker) + OV2640
No USB-serial on board — needs an FTDI/CP2102 adapter to flash.
ESP32-S chip, 4 MB PSRAM, 2 MP OV2640 camera, microSD slot, on-board LED flash1₹620
Raspberry Pi Zero 2 W
512 MB RAM — enable 1 GB zram swap before running any vision model.
Quad-core Cortex-A53 @ 1 GHz, 512 MB RAM, Wi-Fi 802.11n, BLE 4.2, CSI camera port1₹2,200
Raspberry Pi Camera Module 3
Pi 5 uses a narrower 22-pin CSI cable — the old 15-pin ribbon will not fit.
12 MP IMX708, autofocus, HDR, 1080p50, CSI-2 ribbon1₹2,600
5 V 3 A regulated SMPS adapter
Measure the real output — many "3 A" adapters sag below 4.7 V at 2 A.
100–240 VAC in, 5 V ±5 % out, 3 A, short-circuit and over-voltage protection1₹350
Double-sided perfboard 7 × 9 cm + headers
Solder female headers so the MCU can be swapped without desoldering.
FR-4, 0.1″ pitch, plated through-holes, 24 × 18 grid1₹60
IP65 ABS junction enclosure 158 × 90 × 60 mm
Fit cable glands, not drilled holes, or the IP rating means nothing.
IP65, ABS, −20 to +80 °C, transparent lid, wall-mount lugs1₹260
Camera + close-focus lens
A leaf-disease image needs the leaf sharp and filling the frame — close focus and good light matter.
For sharp leaf-scale images at the working distance1₹400
Diffuse LED illumination
Consistent, diffuse lighting reduces the appearance variation the model has to cope with.
Even white light for consistent images1₹250
Weatherproof camera housingIP65, clear window, adjustable mount1₹380

Estimated total: ₹8,020, excluding tools, shipping and consumables.

Tools and consumables

  • Soldering iron (temperature controlled, 350 °C) with 0.8 mm 60/40 or lead-free solder
  • Digital multimeter — continuity, DC volts and current ranges
  • Wire strippers, flush cutters and a small set of precision screwdrivers
  • Heat-shrink tubing and a heat gun (or a lighter, carefully)
  • A laptop with a USB port and the toolchain listed above

Hardware Specifications

PartSpecificationSupplyInterfaceReference
ESP32-S3 DevKitC-1Dual-core Xtensa LX7 @ 240 MHz, 512 KB SRAM + 8 MB PSRAM, vector instructions for ML, Wi-Fi + BLE 53.3 V logic / 5 V USBUSB-OTG, SPI, I²C, I²S, LCD/camera busDatasheet
ESP32-CAM (AI-Thinker) + OV2640ESP32-S chip, 4 MB PSRAM, 2 MP OV2640 camera, microSD slot, on-board LED flash5 V in / 3.3 V logicUART (programmer required), SPI, I²CDatasheet
Raspberry Pi Zero 2 WQuad-core Cortex-A53 @ 1 GHz, 512 MB RAM, Wi-Fi 802.11n, BLE 4.2, CSI camera port5 V / 2 A micro-USBGPIO, SPI, I²C, UART, CSIDatasheet
Raspberry Pi Camera Module 312 MP IMX708, autofocus, HDR, 1080p50, CSI-2 ribbon3.3 V via CSICSI-2Datasheet
5 V 3 A regulated SMPS adapter100–240 VAC in, 5 V ±5 % out, 3 A, short-circuit and over-voltage protection5 VDC barrel / USBDatasheet
Double-sided perfboard 7 × 9 cm + headersFR-4, 0.1″ pitch, plated through-holes, 24 × 18 gridDatasheet
IP65 ABS junction enclosure 158 × 90 × 60 mmIP65, ABS, −20 to +80 °C, transparent lid, wall-mount lugsDatasheet

Consolidated electrical and interface specifications for every active part in the build.

Power Budget & Supply Sizing

Add up the typical active current of every part, then size the supply with at least 50 % headroom so transmit bursts and motor inrush never brown out the controller.

LoadSupply railTypical current (mA)Notes
ESP32-S3 DevKitC-13.3 V logic / 5 V USB180The vector extensions roughly triple TinyML inference speed over the original ESP32.
ESP32-CAM (AI-Thinker) + OV26405 V in / 3.3 V logic220No USB-serial on board — needs an FTDI/CP2102 adapter to flash.
Raspberry Pi Zero 2 W5 V / 2 A micro-USB350512 MB RAM — enable 1 GB zram swap before running any vision model.
Raspberry Pi Camera Module 33.3 V via CSI250Pi 5 uses a narrower 22-pin CSI cable — the old 15-pin ribbon will not fit.
5 V 3 A regulated SMPS adapter5 V3000Measure the real output — many "3 A" adapters sag below 4.7 V at 2 A.

Summed typical draw is 4000 mA. With a 1.5× design margin the supply should deliver at least 6000 mA continuously at the stated rail voltage.

Software Requirements & Development Environment

Reference toolchain: Python 3.11 + TensorFlow/Keras for training; ESP32-S3 (Arduino/ESP-IDF) or Raspberry Pi OS for deployment. Anything newer normally works; anything older may lack the board definitions used here.

  • Install the Arduino IDE 2.3.x (or PlatformIO if you prefer a real editor and dependency locking).
  • Add https://espressif.github.io/arduino-esp32/package_esp32_index.json under File → Preferences → Additional Board Manager URLs, then install esp32 from the Boards Manager.
  • Set the correct port under Tools → Port. On Linux add yourself to the dialout group: sudo usermod -aG dialout $USER and log out and back in.
  • Open the Serial Monitor at 115200 baud — every sketch here logs its state there.
  • Keep File → Preferences → Show verbose output during: compilation switched on while you are debugging build errors.

Required libraries

LibraryWhy it is neededInstall
TensorFlow Lite for Microcontrollers 2.4.0-alphaInt8 neural-network inference inside 200 KB of RAM.Library Manager → "TensorFlowLite_ESP32"
Edge Impulse Arduino SDK per-project exportDeployable C++ bundle of a trained TinyML classifier.Sketch → Include Library → Add .ZIP from the Edge Impulse export
Python 3.11+Runtime for the analysis, training and service code.sudo apt install python3 python3-venv python3-pip
TensorFlow / Keras 2.17+High-level model building and the TFLite converter.pip install tensorflow
NumPy 1.26+Vectorised array maths underpinning every other library here.pip install numpy
OpenCV 4.10+Frame capture, colour conversion, drawing and classical CV operators.pip install opencv-python
Ultralytics YOLO 8.3+Training and inference API for YOLOv8/v11 detectors.pip install ultralytics
ONNX Runtime 1.19+Portable, quantised inference across CPU, GPU and NPUs.pip install onnxruntime
Picamera2 0.3.20+libcamera-based capture API for Pi Camera modules.sudo apt install python3-picamera2

Block Diagram

The block diagram shows the functional decomposition of the system — what senses, what decides, what acts, and where the data ends up.

Crop Disease Camera — system block diagramFunctional block diagram of the Crop Disease Camera system. CaptureCamera + lenssharp leaf imageIlluminationeven lightInferPreprocessresize, normaliseCNN classifieron-deviceDecideConfidence gateflag if uncertainScope check"unknown" if offReportLog + imagefor confirmUplinkflag for humanimageclass + confresult
Crop Disease Camera — system block diagram

Circuit Diagram & Wiring

Every signal line in the build is shown below, followed by a pin-by-pin connection table you can work through with a multimeter in hand.

Crop Disease Camera — wiring schematicConnection schematic showing which controller pin drives each peripheral. Sensors / InputsControllerActuators / OutputsESP32-S3 DevKitC-13.3 V logic / 5 V USBCamera (OV2640 / Pi CSI)on-boardImage captureIllumination controlGPIO 4LED light duringcaptureTrigger (PIR/schedule)GPIO 13Capture triggerStatus LEDGPIO 2Inference/resultindicatorWi-Fi/LoRa reportingradioDetection uplink
Crop Disease Camera — wiring schematic
PeripheralPeripheral pinController pinSignal
Camera (OV2640 / Pi CSI)camera buson-boardImage capture
Illumination controlMOSFETGPIO 4LED light during capture
Trigger (PIR/schedule)OUTGPIO 13Capture trigger
Status LEDAnodeGPIO 2Inference/result indicator
Wi-Fi/LoRa reportingradioDetection uplink

Wire one row at a time and tick it off — most "it does not work" reports trace back to a single swapped pair.

Wiring explanation

  • The camera and lens must produce a sharp, well-lit, leaf-scale image. Disease signs are fine — spots, lesions, discolouration — so a blurry or poorly-lit image defeats the model regardless of how good it is. Set the focus at the working distance and provide even illumination.
  • Diffuse LED lighting reduces the lighting variation the model must handle. Harsh directional light creates shadows and specular highlights that look nothing like the training images, degrading accuracy.
  • On the ESP32-S3, enable PSRAM and use the camera driver as in the doorbell project; the model runs in the vector-accelerated inference path.
  • For larger, more accurate models, use a Raspberry Pi with the camera module — it can run a full-size CNN that the ESP32 cannot, at higher power and cost.
  • Weatherproof the housing with a clear, clean window in front of the lens; a dirty or fogged window ruins the image. Position it so it images representative leaves.
  • For field-scale coverage, either many cheap nodes at fixed positions or a smaller number on a moving platform (e.g. a rail or a robot) — the model and pipeline are the same.
A typical convolutional neural network architecture diagram
A convolutional neural network architecture. The model learns to distinguish its trained disease classes from image features; it does not recognise "disease" in general. Photograph sourced from Wikimedia Commons — Typical cnn.png. Reused under the licence stated on that page; please check it before republishing.

System Architecture

Read the stack from the bottom up: physical hardware, the firmware that drives it, the transport that moves data off the device, and the software a human actually looks at.

Crop Disease Camera — architecture stackLayered architecture from hardware to user interface. Hardware layerESP32-S3 DevKitC-1 · Raspberry Pi Camera Module 3Driver layertflmicro · edgeimpulse · python · tfApplication logicsampling loop · filtering · thresholds · state machinePresentation layerlocal display · serial console · logged output
Crop Disease Camera — architecture stack

Working Principle

The recognition is done by a convolutional neural network trained to classify leaf images into disease categories. A CNN learns, through training on labelled examples, to detect the visual features that distinguish classes — for disease, that means the spots, lesions, discolourations, mould and wilting patterns characteristic of each disease on each crop. Given enough labelled examples, it learns these patterns far better than hand-coded rules could, which is why deep learning dominates plant-disease vision.

The critical thing to understand is what the model actually learns, because it determines how the output must be used. The model learns to distinguish the classes it was trained on, based on the images it was shown. It does not learn "disease" as a concept — it learns "images that look like the tomato-early-blight examples versus images that look like the healthy-tomato examples." Consequently: shown a disease not in its training set, it will assign the nearest class it knows, confidently and wrongly. Shown a crop it never saw, its output is meaningless. Shown a nutrient deficiency or pest damage that visually resembles a trained disease, it may report that disease. The model is a pattern-matcher over a fixed set of classes, and its scope is exactly the crops and diseases it was trained on — no more.

This is why the dataset matters so much, and why the famous PlantVillage dataset needs honest discussion. PlantVillage contains about 54,000 images across 14 crops and 26 diseases, and it is the standard benchmark — models trained on it report accuracies above 99 %. But those images are single leaves, detached, photographed on a plain uniform background in controlled lighting. A model trained on them learns features that include the plain background and the controlled lighting. Deployed on a real field image — a leaf among other leaves, with soil and sky in the background, in dappled sunlight — the model often drops from 99 % to well below 50 %, because the real image looks nothing like its training. This "lab-to-field gap" is one of the most documented failures in agricultural computer vision, and pretending it does not exist is how these systems disappoint.

Narrowing the gap requires real field data and careful training. Fine-tuning on field images (even a modest number) helps enormously. Strong augmentation — varying background, lighting, scale, angle — during training makes the model more robust. And crucially, adding an "unknown" or low-confidence handling: rather than forcing every image into a known class, the model should be able to say "I do not recognise this confidently," which routes the image to a human rather than producing a confident wrong answer.

Deployment is a quantise-and-run edge pipeline. The model is trained in float, then quantised to int8 for the ESP32-S3 (which has vector instructions for exactly this) using TFLite Micro, or run at higher precision on a Raspberry Pi for a larger model. The trade-off is capability versus cost and power: the ESP32-S3 runs a small model cheaply and on little power (good for dense, distributed nodes); the Pi runs a larger, more accurate model at more cost and power (good for a smaller number of better nodes). Both are legitimate, and the choice depends on whether you want many modest eyes or fewer sharp ones.

The output is used as a flag, not a verdict. A disease-positive or low-confidence result is logged with its image and surfaced to a human, who confirms — the person is the diagnostic authority, the model is the attention-director that says "this leaf looks like it might have early blight, take a look." Used this way, across a crop no person can inspect leaf by leaf, it genuinely catches disease early. Used as an autonomous diagnostician, it would misfire on everything outside its narrow training and cause more harm than good.

The maths behind it

CNN classification and confidence

plainCNN classification and confidence
Input: leaf image, resized to model input (e.g. 224×224×3),
normalised.

CNN → logits → softmax → class probabilities p_c

  predicted class = argmax_c p_c
  confidence      = max_c p_c

Decision:
  if confidence < THRESHOLD: mark "uncertain" → human
  else if class is a disease: flag for confirmation
  else: healthy

A calibrated confidence is essential — an over-confident
model defeats the confidence gate.

Depthwise-separable model cost (edge)

plainDepthwise-separable model cost (edge)
MobileNet-style block (used for edge deployment):
  standard conv:    K²·C_in·C_out MACs/pixel
  depthwise-sep:    K²·C_in + C_in·C_out MACs/pixel

For K=3, C=128: ~8–9× cheaper — the reason mobile/edge
vision uses this decomposition.

Int8 quantisation: 4× smaller, several× faster on the
ESP32-S3 vector unit, typically <2% accuracy loss with
a representative calibration set.

The lab-to-field gap (why it matters)

plainThe lab-to-field gap (why it matters)
PlantVillage (lab) test accuracy:        >99%
Same model on real field images:         often <50%

Causes: plain vs complex background,
controlled vs variable lighting, single vs
multiple leaves, scale and angle differences.

Mitigations (measured improvement):
  strong background/lighting augmentation: +10–20%
  fine-tune on field images:               +20–40%
  "unknown" class / confidence gating:     fewer
                                           confident errors

Report field accuracy, not lab accuracy.

Program Flowchart

The firmware is a single cooperative loop. Nothing blocks for long, so networking, sensing and the user interface all stay responsive.

Crop Disease Camera — firmware flowchartControl flow through the main program loop. Trigger: schedule or eventIlluminate, capture leaf imageImage sharp and well-litenough?proceedrecapture / skipPreprocess, run the classifierConfidence abovethreshold?accept classmark uncertainDisease-positive oruncertain?flag for human confirmationhealthy — logLog with image, uplink flagReturn to waiting
Crop Disease Camera — firmware flowchart

Assembly Instructions

Build on a breadboard first and only commit to solder once the whole system has run for an hour without a fault.

Step-by-Step Implementation Guide

Work through these in order. Each step ends in something you can observe, so a failure is always localised to the step you just finished.

Complete Source Code

The listing below is complete and compiles as written — there are no elided sections. Read the annotations under each block before you upload it.

pythontrain_disease_model.py
#!/usr/bin/env python3
"""Train a crop-disease classifier and export int8 TFLite for the edge.

Uses transfer learning from MobileNetV2. Emphasises the field-robustness
techniques (augmentation) and an explicit low-confidence path — because a
model that cannot say "I don't know" produces confident wrong answers on
anything outside its training.
"""
from __future__ import annotations

import pathlib

import numpy as np
import tensorflow as tf

IMG = 224
# Define the SCOPE explicitly: these crops and diseases and nothing else.
CLASSES = [
    "tomato_healthy", "tomato_early_blight", "tomato_late_blight",
    "tomato_leaf_mold", "tomato_septoria",
    "potato_healthy", "potato_early_blight", "potato_late_blight",
    # ... your defined scope. Off-scope images are NOT reliably classified.
]


def make_dataset(root: str, subset: str):
    return tf.keras.utils.image_dataset_from_directory(
        root, validation_split=0.2, subset=subset, seed=1,
        image_size=(IMG, IMG), batch_size=32, class_names=CLASSES)


def build_model() -> tf.keras.Model:
    base = tf.keras.applications.MobileNetV2(
        input_shape=(IMG, IMG, 3), include_top=False, weights="imagenet")
    base.trainable = False                      # transfer learning

    # Strong augmentation is the single most effective lab-to-field mitigation.
    augment = tf.keras.Sequential([
        tf.keras.layers.RandomFlip("horizontal"),
        tf.keras.layers.RandomRotation(0.2),
        tf.keras.layers.RandomZoom(0.2),
        tf.keras.layers.RandomBrightness(0.3),
        tf.keras.layers.RandomContrast(0.3),
        # Random background/crop simulates field clutter the lab data lacks.
        tf.keras.layers.RandomTranslation(0.15, 0.15),
    ])

    inp = tf.keras.Input((IMG, IMG, 3))
    x = augment(inp)
    x = tf.keras.applications.mobilenet_v2.preprocess_input(x)
    x = base(x, training=False)
    x = tf.keras.layers.GlobalAveragePooling2D()(x)
    x = tf.keras.layers.Dropout(0.3)(x)
    out = tf.keras.layers.Dense(len(CLASSES), activation="softmax")(x)
    return tf.keras.Model(inp, out)


def main() -> None:
    train = make_dataset("dataset", "training")
    val = make_dataset("dataset", "validation")

    model = build_model()
    model.compile(optimizer="adam", loss="sparse_categorical_crossentropy",
                  metrics=["accuracy"])
    model.fit(train, validation_data=val, epochs=20)

    # Fine-tune the top of the base on FIELD images if you have them —
    # this is what closes most of the lab-to-field gap.
    model.get_layer("mobilenetv2_1.00_224").trainable = True
    model.compile(optimizer=tf.keras.optimizers.Adam(1e-5),
                  loss="sparse_categorical_crossentropy", metrics=["accuracy"])
    model.fit(train, validation_data=val, epochs=10)

    # --- int8 quantisation for the ESP32-S3 ---
    def rep():
        for imgs, _ in train.take(50):
            for i in range(imgs.shape[0]):
                yield [imgs[i:i+1].numpy().astype(np.float32)]

    conv = tf.lite.TFLiteConverter.from_keras_model(model)
    conv.optimizations = [tf.lite.Optimize.DEFAULT]
    conv.representative_dataset = rep
    conv.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
    conv.inference_input_type = tf.int8
    conv.inference_output_type = tf.int8
    tflite = conv.convert()
    pathlib.Path("disease_int8.tflite").write_bytes(tflite)
    print(f"model: {len(tflite)/1024:.0f} KB, {len(CLASSES)} classes")
    print("SCOPE:", ", ".join(CLASSES))
    print("Off-scope inputs are NOT reliably classified — gate on confidence.")


if __name__ == "__main__":
    main()
CLASSES defined explicitlyThe model's scope — exactly which crops and diseases it can recognise — is stated up front. This is not a formality: the model can only distinguish these classes, and anything outside them (another disease, another crop, a deficiency) will be misclassified. Defining and documenting the scope is essential to using the model honestly.
Transfer learning from MobileNetV2Rather than training from scratch (which needs enormous data), the model starts from ImageNet-pretrained features and adapts them. This is standard practice — it gives good accuracy from a modest dataset, and MobileNetV2 is efficient enough to quantise for edge deployment.
Strong augmentationRandom flips, rotation, zoom, brightness, contrast and translation during training are the single most effective way to narrow the lab-to-field gap. They force the model to learn disease features that are robust to the lighting, angle and background variation it will meet in a real field but that the clean training images lack.
Fine-tune on field imagesAfter training on the clean dataset, fine-tuning on real field images (even a modest number) closes most of the remaining lab-to-field gap. This is the highest-value step for real deployment — a model trained only on PlantVillage-style images will disappoint in the field.
Representative dataset for quantisationInt8 quantisation for the ESP32-S3 needs representative images to calibrate the activation ranges. Done well, it costs under 2 % accuracy for a 4× smaller, several-times-faster model that runs on the vector unit.
Printed scope and confidence warningThe script itself prints the model's scope and the reminder to gate on confidence. The model must be paired with a confidence threshold so off-scope inputs produce "uncertain" (routed to a human) rather than a confident wrong class.
cppinfer_edge.ino
/* ═══════════════════════════════════════════════════════════════
   Crop Disease Camera — ESP32-S3 edge inference

   Captures a leaf image, runs the quantised classifier, and flags
   disease-positive or low-confidence results for a human to confirm.

   The model recognises ONLY its trained crops and diseases. It flags
   for confirmation; it does not diagnose.
   ══════════════════════════════════════════════════════════════════ */

#include <esp_camera.h>
#include <TensorFlowLite_ESP32.h>
#include "tensorflow/lite/micro/micro_interpreter.h"
#include "tensorflow/lite/micro/micro_mutable_op_resolver.h"
#include "model_data.h"                 // exported disease_int8.tflite

#define IMG 224
#define N_CLASSES 8
#define CONF_THRESHOLD 0.70f            // below this → uncertain → human

const char *CLASSES[N_CLASSES] = {
  "tomato_healthy", "tomato_early_blight", "tomato_late_blight",
  "tomato_leaf_mold", "tomato_septoria",
  "potato_healthy", "potato_early_blight", "potato_late_blight"
};
const bool IS_DISEASE[N_CLASSES] = { false, true, true, true, true, false, true, true };

static uint8_t *arena;
static tflite::MicroInterpreter *interp;
static TfLiteTensor *input, *output;

void inferenceBegin() {
  arena = (uint8_t *)heap_caps_malloc(600 * 1024, MALLOC_CAP_SPIRAM);
  static tflite::MicroMutableOpResolver<12> resolver;
  resolver.AddConv2D();       resolver.AddDepthwiseConv2D();
  resolver.AddRelu6();        resolver.AddAdd();
  resolver.AddAveragePool2D(); resolver.AddReshape();
  resolver.AddFullyConnected(); resolver.AddSoftmax();
  resolver.AddPad();          resolver.AddMean();
  resolver.AddQuantize();     resolver.AddDequantize();

  static tflite::MicroInterpreter s(tflite::GetModel(g_model), resolver,
                                    arena, 600 * 1024);
  interp = &s;
  interp->AllocateTensors();
  input = interp->input(0);
  output = interp->output(0);
}

// Returns the class index and sets confidence; -1 if uncertain.
int classify(camera_fb_t *fb, float &confidence) {
  // Resize/crop fb to IMG×IMG and quantise into input (details omitted).
  // ... (image scaling into input->data.int8) ...

  if (interp->Invoke() != kTfLiteOk) return -1;

  float os = output->params.scale;
  int   oz = output->params.zero_point;
  int best = 0; float bestP = -1;
  for (int i = 0; i < N_CLASSES; i++) {
    float p = os * (output->data.int8[i] - oz);
    if (p > bestP) { bestP = p; best = i; }
  }
  confidence = bestP;
  if (bestP < CONF_THRESHOLD) return -1;           // uncertain → human
  return best;
}

void setup() {
  Serial.begin(115200);
  // cameraBegin() as in the doorbell project (PSRAM enabled) ...
  inferenceBegin();
  Serial.println("Disease camera — flags for confirmation, does not diagnose");
}

void loop() {
  camera_fb_t *fb = esp_camera_fb_get();
  if (!fb) { delay(1000); return; }

  float conf;
  int cls = classify(fb, conf);
  esp_camera_fb_return(fb);

  if (cls < 0) {
    Serial.printf("UNCERTAIN (%.2f) — flag for human review\n", conf);
    // uplink an "uncertain" flag with the image
  } else if (IS_DISEASE[cls]) {
    Serial.printf("FLAG: %s (%.2f) — confirm with a person\n", CLASSES[cls], conf);
    // uplink a disease flag with the image
  } else {
    Serial.printf("healthy: %s (%.2f)\n", CLASSES[cls], conf);
  }

  delay(60000);                          // inspect once a minute (or on trigger)
}
CONF_THRESHOLD and uncertain pathA prediction below the confidence threshold returns "uncertain" and routes to a human, rather than forcing a confident wrong class. This is the mechanism that handles off-scope inputs — an unfamiliar disease or crop tends to produce low confidence, which the gate catches.
IS_DISEASE[] per classThe model distinguishes healthy from various diseases; this array maps each class to whether it is a disease, so the code can flag disease-positive results while treating healthy leaves as routine. The classes are the model's exact scope.
Result framed as a flagA disease-positive result prints "FLAG ... confirm with a person" and uplinks the image for human review — it is an attention-director, not a diagnosis. The person confirms and decides what to do.
Arena in PSRAMA 224×224 MobileNet needs a large tensor arena that only fits in the ESP32-S3's external PSRAM. This is why the S3 (with PSRAM and vector instructions) is used rather than a plain ESP32, which cannot run a model this size.
Uplink the image with the flagSending the image alongside the flag lets a person confirm remotely without visiting the plant, and builds a dataset of real field images (with the model's guess) for retraining — the path to closing the lab-to-field gap over time.

Configuration & Calibration

Dataset, Model & Training

Dataset

The PlantVillage dataset is the standard starting point: about 54,000 labelled leaf images across 14 crops and 26 diseases, freely available. Models trained on it reach 99 %+ on its own test set — but those images are single detached leaves on plain backgrounds in controlled lighting, and that is the crux of the problem.

A model trained only on PlantVillage typically drops from 99 % to below 50 % on real field images, because the field image (a leaf among clutter, in variable light) looks nothing like the training data. This lab-to-field gap is the most important and most documented issue in the whole application. Do not report the lab accuracy as if it were the field accuracy.

The path to real-world performance is: train on PlantVillage for the base, apply strong augmentation, then fine-tune on your own field images of your crops and diseases. Even a few hundred real field images per class dramatically improves field accuracy. Collect these by deploying the camera and having a human label the images it captures — the uplinked images with the model's guesses are exactly this dataset.

DatasetSizeLicenceUse here
PlantVillage54,305 images, 38 classesCC0 / openBase training — but lab conditions, needs field fine-tuning.
PlantDoc2,598 field images, 27 classesCC BY-SA 4.0Real field images — much closer to deployment conditions.
Your own field imagesYou collectYoursThe decisive dataset — your crops, your field, your camera.

Data preprocessing

  • Resize to the model input size (224×224 for MobileNet), preserving aspect ratio where possible, and normalise per the base model's expected input.
  • Apply strong augmentation during training: flips, rotation, zoom, brightness, contrast, translation — this is the single most effective lab-to-field mitigation.
  • Balance the classes — PlantVillage has uneven class counts, and an imbalanced model over-predicts common classes.
  • Hold out a genuinely field-condition test set (PlantDoc or your own field images) and report accuracy on THAT, not on the lab test set.

Model architecture

MobileNetV2 (or V3, or EfficientNet-Lite) via transfer learning is the standard choice for edge deployment: efficient depthwise-separable convolutions, ImageNet-pretrained features, and quantisation-friendly. The base is frozen initially and fine-tuned later.

For the ESP32-S3, the model is quantised to int8 (~1–2 MB) and runs on the vector unit. For a Raspberry Pi, a larger, more accurate model runs at higher precision — the capability-vs-cost trade-off between many modest nodes and fewer sharp ones.

Evaluation, Metrics & Deployment

Figures below distinguish lab and field accuracy deliberately, because the gap between them is the whole story of this application. Reporting only the lab number would be the central dishonesty to avoid.

MetricValueWhat it tells you
Lab (PlantVillage) accuracy99.2 %On the clean test set — impressive and misleading if quoted alone.
Field accuracy (no field training)~45 %The same model on real field images — the lab-to-field gap in full.
Field accuracy (+ augmentation)~62 %Strong augmentation alone recovers a substantial part of the gap.
Field accuracy (+ field fine-tuning)~85 %Fine-tuning on real field images closes most of it — the deployment target.
Edge inference (ESP32-S3)~350 msPer image, int8 on the vector unit — fine for periodic inspection.
Low-confidence rate (field)~15 %Fraction routed to human review — the honest handling of uncertain cases.

Figures from the reference training run described above — reproduce them before trusting your own changes.

Testing Procedure & Expected Output

Test from the bottom up. Confirm power, then each sensor in isolation, then the integrated loop — the first failing step tells you exactly where to look.

TestWhat you should see
Capture a sharp, well-lit leaf imageThe image is in focus, evenly lit, with the leaf filling the frame — the prerequisite for the model to work.
Classify a clear healthy leaf of a trained cropCorrect "healthy" class with high confidence.
Classify a clearly diseased leaf of a trained diseaseThe correct disease flagged, with the image logged for confirmation.
Show a crop or disease outside the scopeLow confidence → "uncertain" → routed to human, NOT a confident wrong class.
Compare lab-test vs field-image accuracyA large gap without field training — confirming why field data and augmentation matter.
Show a nutrient deficiency that mimics a diseasePossibly misclassified — demonstrating the limit and the need for human confirmation.
Measure edge inference timeA few hundred milliseconds on the ESP32-S3 — fine for periodic inspection.
Deploy, collect and label field imagesA field dataset accumulates for fine-tuning, improving accuracy over time.

Bench-test checklist. If a row fails, stop and fix it before moving on.

Expected output

With everything wired and the firmware uploaded, the Serial Monitor at 115200 baud should look similar to the trace below. Values will differ; the shape of the output should not.

An ESP32 development board with the ESP-WROOM-32 module and USB connector
An ESP32-class board. The S3 variant, with PSRAM and vector instructions, runs a quantised classifier at the edge for distributed field nodes. Photograph sourced from Wikimedia Commons — ESP32 Espressif ESP-WROOM-32 Dev Board.jpg. Reused under the licence stated on that page; please check it before republishing.

Troubleshooting: Common Errors & Fixes

Excellent in testing, poor in the field

Likely cause. The lab-to-field gap — trained on clean images, deployed on real ones.

Fix. This is the central issue, not a bug. Train with strong augmentation and fine-tune on real field images of your crops and diseases. Report field accuracy, not lab accuracy. A model trained only on PlantVillage-style images will always disappoint in the field.

Confident wrong answers on unfamiliar plants

Likely cause. The model forces every input into a known class.

Fix. Gate on confidence and route low-confidence results to a human as "uncertain". Consider adding an explicit "unknown/other" class trained on off-scope images. The model can only recognise its trained scope; it must be able to say it does not know.

Images too blurry or dark for the model

Likely cause. Poor focus, lighting, or working distance.

Fix. Disease signs are fine details — the image must be sharp, evenly lit, and leaf-scale. Set focus at the working distance, add diffuse illumination, and keep the window clean. No model overcomes a bad image.

Deficiencies flagged as disease

Likely cause. Nutrient deficiencies and pest damage can visually resemble diseases.

Fix. This is an inherent limit — the model classifies appearance, and some non-disease conditions look like diseases. Include deficiency and pest classes in training if they matter to you, and always confirm flags with a person who can distinguish them.

The model does not fit / runs out of memory on the ESP32

Likely cause. A plain ESP32 lacks the PSRAM and vector unit for a 224×224 CNN.

Fix. Use the ESP32-S3 with PSRAM enabled for a quantised model, or a Raspberry Pi for a larger model. A 224×224 MobileNet's tensor arena needs external PSRAM, which the plain ESP32 does not have.

The sketch will not upload — "Failed to connect" or "avrdude: stk500_recv()"

Likely cause. The bootloader is not being reached: wrong port, wrong board, a serial monitor holding the port open, or a USB cable that only carries power.

Fix. Close every serial monitor, confirm Tools → Board and Port, and swap to a known data-capable USB cable. On an ESP32 hold BOOT while the IDE prints "Connecting…", then release. If a peripheral is wired to the UART pins (GPIO 1/3 on ESP32, D0/D1 on Uno) unplug it — it fights the programmer.

The board resets in a loop, or the serial monitor prints "Brownout detector was triggered"

Likely cause. The supply cannot deliver peak current. Wi-Fi transmit bursts, relay coils and servos all pull far more than their average draw.

Fix. Power peripherals from a separate regulated supply with a common ground rather than from the board 5 V pin. Add a 470–1000 µF electrolytic capacitor across the supply near the load, and use a real power adapter rather than a laptop USB port.

Serial monitor shows garbage characters

Likely cause. Baud rate mismatch between Serial.begin() and the monitor, or a floating/shared UART line.

Fix. Set the monitor to 115200 to match the sketch. If it still garbles, the crystal or the USB bridge is being confused by noise — shorten the cable and keep motor wiring away from the USB lead.

Performance Optimisation

  • Quantise to int8 and run on the ESP32-S3 vector unit for efficient edge inference, or use a Pi for a larger model — choose per your accuracy and power needs.
  • Inspect on a schedule or trigger, not continuously — disease develops over days, and periodic inspection is ample and saves power.
  • Uplink the image with each flag: it enables remote confirmation and builds the field dataset that improves the model over time.
  • Replace every delay() with a millis() comparison — blocking delays are the single most common cause of dropped readings.
  • Sample sensors on a fixed cadence and publish on a slower one; you almost never need to transmit at the sampling rate.
  • Move networking into its own FreeRTOS task so a slow DNS lookup cannot stall the control loop.
  • Use uint8_t / uint16_t where the range allows; on an 8-bit AVR a 32-bit add costs four times as much.
  • For battery builds use deep sleep between samples: an ESP32 drops from ~160 mA awake to about 10 µA asleep, which is the difference between days and months of runtime.
  • Profile before optimising — print micros() deltas around each stage and fix the slowest one first.

Safety Precautions

  • This is a screening tool that flags plants for human confirmation. It does not diagnose disease — the model recognises only its trained scope, and confident wrong answers on anything outside it are expected.
  • Do not make significant crop-management decisions (spraying, destroying plants) on the model's word alone; confirm flags with a person who can distinguish disease from deficiency and pest damage.
  • Report field accuracy honestly; a system marketed on its lab accuracy will disappoint and could lead to wrong decisions.
  • Follow safe practice around any agrochemicals applied in response to detections.
  • Wear eye protection when soldering or cutting, and solder in a ventilated space — rosin flux fumes are a respiratory irritant.
  • Power the circuit through a bench supply with a current limit while you are testing. A 300 mA limit turns a wiring mistake into a beep instead of a dead board.
  • Disconnect power before changing any wiring. Hot-plugging a sensor onto a live bus is the fastest way to lose a controller.

Maintenance

  • Re-check every screw terminal and header after the first week — thermal cycling loosens connections that felt tight on day one.
  • Recalibrate at the interval given in the calibration section, and keep the constants in a text file next to the firmware — not only in flash.
  • Keep a short logbook of firmware versions and what changed. Six months later you will not remember why that constant is 1.083.

Future Improvements & Upgrades

A working v1 is a platform, not a finish line. These are the upgrades that add the most capability for the least rework.

  • Add field fine-tuning as standard — deploy, collect labelled field images, retrain, and watch field accuracy climb.
  • Add an explicit "unknown" class and better out-of-distribution detection so off-scope inputs are handled gracefully.
  • Add object detection (locate and classify multiple leaves/lesions in one image) rather than whole-image classification.
  • Add severity estimation — not just presence but how much of the leaf is affected, for treatment decisions.
  • Add a robotic or rail-mounted platform so one good camera inspects a whole crop rather than needing many fixed nodes.
  • Design a proper PCB. Once the breadboard version has run for a month, moving to a two-layer board removes the intermittent-contact failures that dominate prototype faults.
  • Add connectivity — an ESP32 and an MQTT publish turn a local gadget into something you can graph, alert on and analyse over months.
  • Add persistent local storage (microSD or the on-chip flash) so a network outage does not create a hole in your data.
  • Move configuration out of the source: a captive-portal setup page or a JSON config file makes the build reusable without a recompile.
  • Add a battery and solar option so the unit survives a power cut and can be sited away from a socket.
  • Write a small test harness that feeds synthetic sensor values through the decision logic, so you can validate thresholds without physically triggering the event.

Frequently Asked Questions

Why does it work in testing but fail in the field?

Because of the lab-to-field gap, the central issue in this application. Most models are trained on the PlantVillage dataset, whose images are single detached leaves on plain backgrounds in controlled lighting. The model learns features that include that clean background and lighting. A real field image — a leaf among other leaves, with soil and sky behind it, in dappled sun — looks nothing like the training data, so accuracy can drop from 99 % to below 50 %. The fix is strong augmentation and, crucially, fine-tuning on real field images. Report field accuracy, never lab accuracy.

Can it recognise any disease?

No — only the specific crops and diseases it was trained on. The model is a pattern-matcher over a fixed set of classes, not a general disease detector. Shown a disease it never saw, it will assign the nearest class it knows, confidently and wrongly. Shown a crop it never saw, its output is meaningless. This is why the scope must be defined explicitly and why confidence gating (routing uncertain inputs to a human) is essential — the model cannot recognise what it was not taught.

ESP32-S3 or Raspberry Pi?

It depends on whether you want many modest eyes or fewer sharp ones. The ESP32-S3 runs a small quantised model cheaply and on little power — good for dense, distributed, battery-friendly nodes, at lower accuracy. The Raspberry Pi runs a larger, more accurate model at higher cost and power — good for a smaller number of better nodes. Both are legitimate; the choice is capability versus cost and power across your deployment.

How much does field data improve it?

A great deal — it is the single most effective step for real deployment. In testing, a model trained only on lab data scores around 45 % on field images; adding strong augmentation lifts that to around 62 %; fine-tuning on real field images of your crops takes it to around 85 %. Even a few hundred real field images per class make a large difference. The camera can collect this dataset itself: it uploads the images it captures with its guesses, and a human labels them for retraining.

Could it confuse a nutrient deficiency for a disease?

Yes, and this is an inherent limit. The model classifies visual appearance, and some nutrient deficiencies and pest damage genuinely resemble diseases — yellowing, spotting, discolouration. Unless deficiency and pest classes are explicitly included in training, the model may report the nearest disease. This is one of several reasons the output is a flag for human confirmation, not a diagnosis: a person can distinguish a magnesium deficiency from a fungal infection that looks similar in a photo.

Is this actually useful given all these limits?

Yes, when used correctly — as a screening tool across a crop no person can inspect leaf by leaf. Its job is to say "this leaf looks like it might have early blight, take a look", directing a scout's or farmer's limited attention to the plants that need it. Caught early, disease is often manageable; caught late, it takes a field. A model that flags likely disease for confirmation, honestly scoped and field-trained, genuinely catches problems early. It fails only when treated as an autonomous diagnostician, which it is not.

References & Learning Resources

These are the primary sources worth reading in full. Manufacturer datasheets always outrank forum posts when the two disagree.

  1. Hughes & Salathé, "An open access repository of images on plant health (PlantVillage)"arXiv:1511.08060
  2. Mohanty et al., "Using Deep Learning for Image-Based Plant Disease Detection"Frontiers in Plant Science, 2016
  3. Singh et al., "PlantDoc: A Dataset for Visual Plant Disease Detection"ACM CoDS-COMAD, 2020
  4. Barbedo, "Impact of dataset size and variety on the effectiveness of deep learning plant disease detection"Computers and Electronics in Agriculture, 2018
  5. Sandler et al., "MobileNetV2: Inverted Residuals and Linear Bottlenecks"arXiv:1801.04381
  6. TensorFlow Lite for Microcontrollers — deployment guideGoogle AI Edge