← All workMachine learning researchLive demo2026

Tomato leaf disease detection: hybrid CNN-Transformer

Runs on free hosting, so the first load can take up to a minute.

Recorded from the live app with a field photo
On this page
  1. In short
  2. The problem
  3. What I built
  4. The hard parts
  5. Recognition and role
  6. Links
  7. Impact

In short

  • WhatPhone photos of tomato leaves, classified into ten conditions.
  • HowA hybrid CNN-Transformer, hardened for low light and budget cameras.
  • ResultLow-light F1 rose from 0.42 to 0.78, a 36-point gain.

The problem

Published plant-disease detection models score 95 to 99 percent on clean laboratory images. But smallholder farmers in Zimbabwe and sub-Saharan Africa take photos under real field conditions: low light, cheap-phone cameras with 5MP sensors and no optical image stabilisation, motion blur, and cluttered backgrounds. Under those conditions the same models collapse, and these farmers rarely have access to an agronomist.

This project is my completed BSc Honours dissertation and the proof of concept beneath a larger multi-crop research programme, ZimCropGuard. It is finished and deployed as a live demo.

What I built

A hybrid CNN-Transformer in PyTorch that classifies ten tomato leaf conditions from a single phone photo, deliberately engineered and measured for robustness to real-world image degradation rather than only reporting clean-lab accuracy. It is a Transformer-augmented CNN, not a Vision Transformer.

Hybrid CNN-Transformer architectureA ResNet18 feature extractor turns a leaf photo into a 512 by 7 by 7 feature map. Two branches read it: a CNN global branch that average-pools and projects to 256 dimensions, and an attention branch that turns the 49 spatial cells into 256-dimensional tokens with a CLS token and positional embeddings, passed through a 4-layer Transformer encoder with 8 heads, pre-norm and GELU. Late fusion concatenates the two into 512 dimensions, and an MLP head predicts the class.Leaf photoone phone imageResNet18ImageNet-pretrainedfeatureextractorFeature map512 × 7 × 7CNN globalbranchglobal averagepool, projectedto 256Attentionbranch49 spatialtokens projectedto 256, with aCLS token andpositionalembeddingsTransformerencoder4 layers, 8heads, pre-norm,GELULate fusion512: globalvector plus CLSoutputMLP headten tomato leafclassesHybrid CNN-Transformer architectureA ResNet18 feature extractor turns a leaf photo into a 512 by 7 by 7 feature map. Two branches read it: a CNN global branch that average-pools and projects to 256 dimensions, and an attention branch that turns the 49 spatial cells into 256-dimensional tokens with a CLS token and positional embeddings, passed through a 4-layer Transformer encoder with 8 heads, pre-norm and GELU. Late fusion concatenates the two into 512 dimensions, and an MLP head predicts the class.Leaf photoone phone imageResNet18ImageNet-pretrainedfeature extractorFeature map512 × 7 × 7CNN global branchglobal average pool,projected to 256Attention branch49 spatial tokensprojected to 256, witha CLS token andpositional embeddingsTransformer encoder4 layers, 8 heads,pre-norm, GELULate fusion512: global vector plusCLS outputMLP headten tomato leaf classes

Architecture as implemented in model.py: a Transformer-augmented CNN, not a Vision Transformer.

Technical detail

The architecture (from the model code, treated as ground truth).

  • An ImageNet-pretrained ResNet18 backbone acts as the feature extractor, producing a 512-channel 7x7 feature map.
  • Two parallel branches read that map. The CNN-global branch applies global average pooling and a linear projection to a 256-dimensional vector. The attention branch flattens the 49 spatial cells into 49 tokens, projects each to 256 dimensions, and prepends a learnable CLS token and positional embeddings.
  • A 4-layer Transformer encoder (8 attention heads, GELU MLP, pre-norm residual blocks) reasons over those tokens. The embedding dimension is 256 throughout.
  • Late fusion concatenates the CNN global vector with the Transformer CLS output into a 512-dimensional fused representation, which an MLP classifier head maps to the ten classes.

Trained for the field, not the lab. The same model is retrained with field-simulating augmentation, so any gain comes from the training strategy alone. It is tested on real farm photos from three tiers of phone, labelled by an independent agricultural specialist, and runs as a live demo in any phone browser.

Technical detail

Two-stage training design. A baseline model uses standard augmentation; a hardened model is retrained with aggressive field-simulating augmentation (brightness variation, contrast variation, Gaussian noise and blur, perspective transforms, resolution simulation), MixUp (alpha 0.2), and label smoothing (epsilon 0.1). The architecture and hyperparameters stay identical, so any accuracy gain is attributable to the training strategy alone, not the model. This is the core scientific contribution.

Structured field-evaluation protocol. 94 real farm photos across 3 device tiers (budget, mid-range, flagship) and 2 lighting conditions, with disease labels assigned independently by an agricultural specialist. The device tiers were pre-registered from camera hardware specifications before any results were seen, so the evaluation separates model failure from sensor failure rather than confirming itself.

Live demo. A Streamlit web app on Streamlit Community Cloud, public and free to try. Upload a photo and receive the predicted class, confidence score, top-5 probabilities, photo tips and a model card. No app install is required, so it works from any phone browser.

The hard parts

I closed the lab-to-field gap through training strategy rather than a bigger model, and designed the evaluation so it cannot quietly confirm itself.

Technical detail

Closing the lab-to-field gap through training strategy alone. The baseline performed well on clean data but failed on degraded field images. Rather than swapping in a larger model, I fixed it through augmentation, MixUp, and label smoothing, producing a cleaner, more defensible result and an explicit point of difference from the GAN-based base paper.

Designing out circular validation bias. Field labels came from an independent agricultural specialist and were not model-assisted. Device tiers were defined by camera hardware before any results were seen. The evaluation cannot quietly confirm itself.

Separating model failure from sensor failure. The field accuracy gradient (64 percent budget, 81.5 percent mid-range, 90.5 percent flagship) is interpreted honestly as a hardware and image-quality effect, not hidden or glossed over as most write-ups do.

Recognition and role

This is the completed dissertation and the deployed proof of concept. The multi-crop research it seeds, ZimCropGuard, is the work that won Best Poster at Deep Learning IndabaX Zimbabwe 2026 and was selected to represent Zimbabwe at the Deep Learning Indaba 2027. I was also featured in ITU News as one of the people powering machine-learning solutions globally, where I shared the advice I keep coming back to: “Do not wait until you feel fully ready to participate.”

Impact

  • Clean-test accuracy: baseline 95 percent, hardened 93 percent (only 2 points lower)
  • Simulated low-light F1: 0.42 to 0.78, a gain of 36 points, which is the headline result
  • Field accuracy by device tier: 64 percent (budget), 81.5 percent (mid-range), 90.5 percent (flagship)
  • Completed BSc Honours dissertation, deployed as a live demo on Streamlit Community Cloud on free CPU inference
Try the live demo ↗Runs on free hosting, so the first load can take up to a minute.

Want to talk about this work? Email me or download my CV.