Automated Pollen Recognition in Optical and Holographic Microscopy Images

AI summary

Overview

Research area: Computer vision applied to microscopy — deep learning for detecting and classifying pollen grains in conventional RGB optical microscopy images and in greyscale lensless digital holographic microscopy (DHM) images, motivated by veterinary cytology.

Technical level: Intermediate. The paper assumes familiarity with object detection and image classification pipelines, transfer learning, and evaluation metrics such as mAP50 and top-line accuracy, but the experimental logic is described in accessible terms.

Scope: A single study comparing two compact deep learning models (YOLOv8s for detection, MobileNetV3L for classification) on 20 RGB optical and 11 greyscale holographic microscopy images of pollen grains, plus dataset-expansion and bounding-box-expansion strategies aimed at closing the cross-modality performance gap.

What This Paper Is About

Lensless digital holographic microscopy is far cheaper and simpler than conventional optical microscopy, which could make diagnostic microscopy available in poorly connected or less developed areas. The problem is that machine learning models trained on RGB optical images of pollen grains transfer poorly to the greyscale holograms produced by these cheaper devices. The paper measures that gap and tests two remedies — automated labeling to grow the training set, and enlarging bounding boxes — to see how much of the loss can be recovered.

Key Contributions

1. A direct cross-modality benchmark for pollen analysis: the same YOLOv8s detection model and MobileNetV3L classification model were trained and evaluated on both RGB optical and greyscale holographic microscopy images of pollen grains from veterinary cytology slides, over four pollen classes (T1, T2, T5, T9).

2. A label-transfer pipeline via image registration: because humans struggle to identify pollen grains in greyscale holograms, the authors aligned the 3840 by 2160 pixel holographic images (average file size 15.8 megabytes) to the optical images (averaging 17500 by 8000 pixels, average file size 55 megabytes) using a similarity transformation and reused the optical bounding-box labels unchanged on the holographic images.

3. Automated labeling at scale: a YOLOv8s model trained on a manually labeled subset generated class-agnostic bounding boxes, expanding annotated instances by factors of 13.82, 15.05, 24.26, and 25.85 across the four dataset variants.

4. Quantified bounding-box expansion strategy: testing 25% and 50% area expansion on holographic images, alone and combined with automated labels, and reporting which combinations helped detection versus classification.

Main Findings

• Optical images set a strong baseline: YOLOv8s reached 91.3% mAP50 for detection and MobileNetV3L reached 97% overall accuracy for classification on RGB optical images.

• Automated labeling transformed optical detection: mAP50 rose from 46.2% with manual labels (5590 object instances) to 91.3% with automated labels (77268 object instances), a 1.97 times improvement from a 13.82 times larger effective dataset.

• Automated labeling did not change optical classification: overall accuracy stayed at 97% when moving from 4536 manual instances to 68268 automated instances, a 15.05 times larger dataset, which the authors describe as performance plateauing.

• The modality gap is severe: shifting from optical to holographic images with manual labels dropped detection mAP50 by 18.55 times, from 46.2% to 2.49%, and dropped classification accuracy by 2.30 times, from 97% to 42%.

• Detection on holograms improved with automated labels: mAP50 went from 2.49% to 8.15% (3.27 times) using 71639 automated instances instead of 2952 manual instances, a 24.26 times larger effective dataset.

• Bounding-box expansion helped detection: 25% expansion raised mAP50 from 2.49% to 4.32% (1.73 times) with manual labels; 50% expansion raised it from 2.49% to 5.53% (2.22 times).

• The best holographic detection result combined both remedies: automated labels plus 50% expansion reached 13.3% mAP50, a 1.63 times increase over automated labels alone (8.15%). Automated labels plus 25% expansion reached 10.2% mAP50.

• Classification on holograms improved but plateaued lower: accuracy went from 42% to 50% with automated labels (1.19 times, from a 25.85 times larger dataset, 63018 instances instead of 2437), from 42% to 53% with 25% expansion (1.26 times), and from 42% to 54% with 50% expansion (1.28 times).

• Combining remedies was not uniformly additive for classification: automated labels plus 25% expansion gave 51%, a 1.03 times decrease relative to 25% expansion alone (53%) and a 1.02 times increase relative to automated labels alone (50%). Automated labels plus 50% expansion stayed at 54%, unchanged from 50% expansion alone, but a 1.05 times improvement over the 51% obtained with 25% expansion plus automated labels.

• The gap was not closed: the authors state that a performance gap of 78% in mAP50 and 43% in overall accuracy between the optical and holographic modalities remains for automated analysis in microscopy imaging.

• Detection suffered more than classification: the paper notes the drop in detection performance was especially large compared to the drop in classification performance, and that understanding why is an open question.

• Class imbalance was present and addressed: the holographic data was uneven across the four pollen classes (manual labels: 518 for T1, 105 for T2, 313 for T5, 1501 for T9), which the authors countered with augmentation, inverse-frequency class weighting, and a weighted cross-entropy loss.

Methodology in Plain English

The researchers collected microscopy images of sample slides containing pollen grains, fixed using standard veterinary cytology techniques with adhesive tape. Twenty high-resolution RGB optical images came from a conventional brightfield microscope paired with an off-the-shelf digital slide scanner (the Ocus 20 by Fisher Scientific), and eleven high-resolution greyscale holographic images came from a custom-made lensless DHM device. Data was split 70:15:15 into training, validation, and testing.

Because holograms are hard for humans to annotate, the team registered each holographic image onto its optical counterpart and copied the optical bounding boxes across. Registration was done as a six-step similarity transformation: computing centroids of the point sets, subtracting the centroid from each set, calculating the scale relationship between sets, normalizing the demeaned points onto a common scale, deriving the optimal rotation matrix with Singular Value Decomposition, and combining scale, rotation, and translation into one transformation matrix. Purely black edge regions introduced by the aspect-ratio difference were handled by screening images with an automated script and excluding any image with 20% or more black area.

To grow the labeled data, a YOLOv8s model was trained on a manually labeled subset to output class-agnostic pollen bounding boxes, creating automated labels that multiplied the instance counts. Both models were then trained in a fixed order: optical images first, holographic second; manual labels first, automated second; and finally with all bounding boxes enlarged by 25% and 50%.

Detection used YOLOv8s pre-trained on COCO, with images cut into 640 by 640 pixel pieces, 200 epochs, uniform learning rate adjustment, and early stopping patience of 25 epochs. Augmentation was deliberately restrained to preserve spatial relationships: rotation up to 45 degrees, vertical flipping with 50% probability, and mixup blending with 10% probability. Class imbalance was handled by YOLOv8s's built-in focal loss and composite loss, and mAP at an intersection-over-union threshold of 0.5 was the headline metric.

Classification used MobileNetV3L pre-trained on ImageNet, with crops ranging from 75 to 150 pixels resized to 112 by 112 pixels — half the model's default size for maximum compatibility. Training was two-phase: 30 epochs with a frozen backbone for transfer learning, then 30 epochs with the final 20 layers unfrozen, with early stopping patience of 10 epochs and dropout of 0.5 on the classifier head. Augmentation was heavier: rotation up to 40 degrees, horizontal flipping at 50% probability, affine translation up to 20%, random resized cropping retaining about 80–100% of content, and intensity/color jitter of plus or minus 20% on brightness, contrast, and saturation plus plus or minus 10% on hue. Class imbalance was handled with inverse-frequency class weights applied to a weighted cross-entropy loss, and overall accuracy was the headline metric. The choice to use two separate models rather than YOLOv8s's own classification head was explained by architectural specialization — MobileNetV3L is built for fine-grained discrimination, while YOLOv8s's classification head favors speed inside the detection pipeline. All experiments used Python and the Jupyter development environment, and both models were chosen with edge-device deployment in mind.

Why This Matters

The work quantifies how badly vision models degrade when moved from the color optical images that dominate microscopy datasets to the cheap greyscale holograms that could democratize microscopy. It shows that automated labeling and bounding-box expansion measurably recover part of that loss — for detection, mAP50 went from 2.49% to 13.3%; for classification, accuracy went from 42% to 54% — while making clear that neither trick closes the gap. The authors' own framing is that the results demonstrate the difficulty of automatic object recognition in holographic images, and that pairing deep learning with cost-effective lensless DHM is plausible at least for image classification tasks.

Real-world applications:

• Veterinary diagnostics: recognizing pollen types associated with adverse immune responses caused by overexposure, and supporting diagnosis of health conditions linked to pollen exposure in animals.
• Environmental and agricultural monitoring: pollen grains are involved in seasonal allergies, species spread, weather changes, crop yields, natural vegetation patterns, and climate change, so accessible pollen recognition supports professionals in those fields.
• Accessibility of microscopy services: lensless DHM is described as dramatically lower in cost and simpler to set up than conventional optical microscopy, potentially bringing microscopic analysis to underserved and resource-constrained regions.
• Edge deployment: both models were selected for compactness and state-of-the-art performance on their tasks, with deployment on edge devices an explicit goal of the study.

Industry relevance: the authors include veterinary and cytology company affiliations (Vetamplify SIA and VetCyto SIA in Riga, Latvia), and the work is funded by the Latvian Council of Science under project VetCyto, project Nr. lzp-2023/1-0220. For device makers and diagnostic service providers, the results indicate that classification on low-cost holographic hardware is the nearer-term opportunity, while detection requires further engineering. The paper also cites prior veterinary pollen classification reaching an average accuracy of 88%, positioning this work relative to existing veterinary automation efforts.

Future Directions

• Build training pipelines or model architectures that better accommodate the characteristic features of greyscale holographic images, which the authors identify as the priority for future work.

• Improve domain adaptation techniques on top of the automated-labeling and bounding-box-expansion approach presented here.

• Explain why the detection drop is so much larger than the classification drop — the paper explicitly states this question still needs to be understood.

• Verify whether image alignment contributed to the performance gap. The authors judge alignment was most probably not a significant factor, but they acknowledge it could have had an impact; holographic images were also upsampled to match their larger optical counterparts, and images with 20% or more black area were excluded after alignment.

Target Audience

Researchers and practitioners in computer vision for microscopy, digital holography, and computational imaging who are interested in cross-modality transfer; veterinary and clinical diagnosticians exploring automation of cytology workflows; developers building edge-deployable detection and classification models for low-cost lensless DHM devices; and agricultural, environmental, and allergy researchers who need scalable pollen identification.

Authors’ abstract

This study explores the application of deep learning to improve and automate pollen grain detection and classification in both optical and holographic microscopy images, with a particular focus on veterinary cytology use cases. We used YOLOv8s for object detection and MobileNetV3L for the classification task, evaluating their performance across imaging modalities. The models achieved 91.3% mAP50 for detection and 97% overall accuracy for classification on optical images, whereas the initial performance on greyscale holographic images was substantially lower. We addressed the performance gap issue through dataset expansion using automated labeling and bounding box area enlargement. These techniques, applied to holographic images, improved detection performance from 2.49% to 13.3% mAP50 and classification performance from 42% to 54%. Our work demonstrates that, at least for image classification tasks, it is possible to pair deep learning techniques with cost-effective lensless digital holographic microscopy devices.