Tomato leaf disease detection: hybrid CNN-Transformer

The problem

Published plant-disease detection models score 95 to 99 percent on clean laboratory images. But smallholder farmers in Zimbabwe and sub-Saharan Africa take photos under real field conditions: low light, cheap-phone cameras with 5MP sensors and no optical image stabilisation, motion blur, and cluttered backgrounds. Under those conditions the same models collapse, and these farmers rarely have access to an agronomist.

This project is my completed BSc dissertation and the proof of concept beneath a larger multi-crop research programme, ZimCropGuard. It is finished, deployed, and live.

What I built

A hybrid CNN-Transformer in PyTorch that classifies ten tomato leaf conditions from a single phone photo, deliberately engineered and measured for robustness to real-world image degradation rather than only reporting clean-lab accuracy. It is a Transformer-augmented CNN, not a Vision Transformer.

The architecture (from the model code, treated as ground truth).

An ImageNet-pretrained ResNet18 backbone acts as the feature extractor, producing a 512-channel 7x7 feature map.
Two parallel branches read that map. The CNN-global branch applies global average pooling and a linear projection to a 256-dimensional vector. The attention branch flattens the 49 spatial cells into 49 tokens, projects each to 256 dimensions, and prepends a learnable CLS token and positional embeddings.
A 4-layer Transformer encoder (8 attention heads, GELU MLP, pre-norm residual blocks) reasons over those tokens. The embedding dimension is 256 throughout.
Late fusion concatenates the CNN global vector with the Transformer CLS output into a 512-dimensional fused representation, which an MLP classifier head maps to the ten classes.

Two-stage training design. A baseline model uses standard augmentation; a hardened model is retrained with aggressive field-simulating augmentation (brightness variation, contrast variation, Gaussian noise and blur, perspective transforms, resolution simulation), MixUp (alpha 0.2), and label smoothing (epsilon 0.1). The architecture and hyperparameters stay identical, so any accuracy gain is attributable to the training strategy alone, not the model. This is the core scientific contribution.

Structured field-evaluation protocol. 94 real farm photos across 3 device tiers (budget, mid-range, flagship) and 2 lighting conditions, with disease labels assigned independently by an agricultural specialist. The device tiers were pre-registered from camera hardware specifications before any results were seen, so the evaluation separates model failure from sensor failure rather than confirming itself.

Live deployment. A Streamlit web app on Streamlit Community Cloud. Upload a photo and receive the predicted class, confidence score, top-5 probabilities, and disease-management guidance. No app install is required, so it works from any phone browser.

The hard parts

Closing the lab-to-field gap through training strategy alone. The baseline performed well on clean data but failed on degraded field images. Rather than swapping in a larger model, I fixed it through augmentation, MixUp, and label smoothing, producing a cleaner, more defensible result and an explicit point of difference from the GAN-based base paper.

Designing out circular validation bias. Field labels came from an independent agricultural specialist and were not model-assisted. Device tiers were defined by camera hardware before any results were seen. The evaluation cannot quietly confirm itself.

Separating model failure from sensor failure. The field accuracy gradient (64 percent budget, 81.5 percent mid-range, 90.5 percent flagship) is interpreted honestly as a hardware and image-quality effect, not hidden or glossed over as most write-ups do.

Recognition and role

This is the completed dissertation and the deployed proof of concept. The multi-crop research it seeds, ZimCropGuard, is the work that won Best Poster at IndabaX Zimbabwe 2026 and was selected to represent Zimbabwe at the Deep Learning Indaba 2027. I was also featured in ITU News as one of the people powering machine-learning solutions globally, where I shared the advice I keep coming back to: “Do not wait until you feel fully ready to participate.”

Impact

Clean-test accuracy: baseline 95 percent, hardened 93 percent (only 2 points lower)
Simulated low-light F1: 0.42 to 0.78, a gain of 36 points, which is the headline result
Field accuracy by device tier: 64 percent (budget), 81.5 percent (mid-range), 90.5 percent (flagship)
Completed BSc dissertation, deployed live on Streamlit Community Cloud on free CPU inference