ForCM: Forest Cover Mapping from Multispectral Sentinel-2 Image by Integrating Deep Learning with Object-Based Image Analysis

AI summary

Overview

Research area: Remote sensing and computer vision — forest cover mapping from satellite imagery, specifically combining deep learning (semantic segmentation) with Object-Based Image Analysis (OBIA).

Technical level: Intermediate. The paper uses well-known segmentation architectures (UNet family, ResNet50-SegNet) and standard GIS tools (QGIS, Orfeo ToolBox), so readers need some familiarity with semantic segmentation and geospatial image processing to follow the details, but the overall workflow is described step by step.

Scope (one sentence): The paper evaluates five deep learning segmentation models on multispectral Sentinel-2 Level-2A imagery of the Amazon Rainforest, then fuses the two strongest models' prediction heatmaps with a traditional OBIA pipeline (mean shift segmentation + SVM classification) in QGIS to test whether the hybrid improves forest/non-forest mapping over OBIA alone.

What This Paper Is About

Accurate forest cover maps are needed to track deforestation and support conservation, but existing pixel-based deep learning methods struggle with precise object boundaries, while object-based methods (OBIA) depend on image quality, careful segmentation parameter tuning, and often on costly specialized software. The paper proposes "ForCM" (Forest Cover Mapping), which feeds heatmaps produced by deep learning models into an OBIA segmentation-and-classification workflow so that the two approaches compensate for each other's weaknesses. The goal is to raise mapping accuracy while keeping the whole pipeline buildable with free, open-source tools.

Key Contributions

1. A DL-plus-OBIA fusion pipeline (ForCM) that combines deep learning probability heatmaps with spectral features extracted from image objects, using mean shift segmentation and an SVM classifier implemented through QGIS 3.34.5 "Maidenhead" and the Orfeo Toolbox Provider (OTB) 8.1.2 plugin.
2. A comparative evaluation of five segmentation architectures — UNet, UNet++, ResUNet, Attention UNet, and ResNet50-SegNet — trained and tested under identical conditions on four Amazon Rainforest Sentinel-2 datasets (3-band V1, 3-band V2, 3-band V3, and 4-band).
3. A direct benchmark against traditional OBIA, using the same randomly selected images and the same mean shift and SVM parameters, measuring IoU, overall accuracy, precision, recall, and F1 score.
4. A demonstration that competitive mapping results are achievable with free, non-commercial tooling, addressing the cost and accessibility barriers of commercial GIS software for forest monitoring.

Main Findings

• DL model behavior across datasets: Attention UNet achieved the strongest results on the 4-band dataset (IoU 0.9168, accuracy 95.47%, F1 0.9581, test accuracy 95.93%), while UNet and UNet++ reached high IoU on the 3-band datasets (UNet 0.9111 on 3-band V3; UNet++ 0.9074 on 3-band V2). ResNet50-SegNet was generally the weakest of the five, with IoU 0.8827 and 0.8826 on 3-band V2 and V3.

• Sensitivity to band composition: UNet performed well on 3-band data but dropped to IoU 0.7142 and test accuracy 81.28% on the 4-band dataset, showing that adding the NIR band did not help every architecture.

• Model selection for fusion: Based on accuracy metrics, training/validation curves, prediction mask comparisons, computational and time complexity, and prior literature, ResUNet and Attention UNet were chosen to generate heatmaps for the OBIA integration.

• ForCM beats traditional OBIA: Traditional OBIA reached IoU 0.8992, overall accuracy 92.91%, precision 92.43%, recall 95.12%, and F1 0.9365. ResUNet-OBIA improved IoU to 0.9101, overall accuracy to 94.54%, precision to 0.9369, recall to 0.9698, and F1 to 0.9525. Attention UNet-OBIA achieved the highest overall accuracy at 95.64% (IoU 0.9064, precision 93.32%, recall 96.84%, F1 0.9504).

• Different models win on different metrics: Attention UNet-OBIA produced the best overall accuracy (95.64%), while ResUNet-OBIA produced the best IoU (0.9101), precision (0.9369), recall (0.9698), and F1 (0.9525). The paper concludes that ResUNet-OBIA significantly outperforms basic OBIA overall.

• Visual confirmation: Side-by-side comparisons of OBIA, ResUNet-OBIA, Attention UNet-OBIA predictions against ground truth masks showed that the integrated methods produced classifications closer to the reference masks than OBIA alone.

• Acknowledged limitations: Performance is sensitive to OBIA parameter settings (spatial radius 5, range radius 5, minimum segment size chosen by trial and error; SVM with linear kernel and regularization parameter 1), and depends on image quality. The authors note that paid GIS software offering multi-resolution segmentation could yield better segmentation than the tools used here.

Methodology in Plain English

Data. The study uses open-source Sentinel-2 Level-2A imagery of the Amazon Rainforest at 10 m spatial resolution, covering four spectral bands (Red, Blue, Green, NIR), all obtained from Zenodo. Four dataset configurations are used: 3-band V1 (30 training / 15 validation / 15 test images), 3-band V2 (1123 / 100 / 100), 3-band V3, which combines selected images from V1 and V2 (280 / 115 / 100), and a 4-band dataset (499 / 100 / 20). All images are 512 × 512 GeoTIFFs; ground truth masks are PNG files for the 3-band datasets and GeoTIFF for the 4-band dataset. Because of computational cost, only 250 images from the 3-band V2 dataset were actually selected for training.

Preprocessing. For the 3-band data, pixel values are divided by 255 and reshaped to (512, 512, 3) as float32; masks have 1 subtracted from pixel values, are reshaped to (512, 512, 1), and cast to int. The 4-band images are converted to float32 and masks are shaped to (1, 512, 512, 1).

Training setup. Experiments ran on Python 3.9.12 with Keras on TensorFlow 2.10.0, using an NVIDIA GeForce GTX TITAN X 12 GB GPU and 32 GB RAM. All models were trained from scratch under identical conditions, with binary cross entropy loss and sigmoid activation. The 3-band V1 models ran for 20 epochs; the other datasets ran for 10 epochs.

Model selection. The five architectures were compared on accuracy, precision, recall, F1 score, complexity, loss graphs, computation time, and visual mask quality. ResUNet and Attention UNet were carried forward.

The OBIA side. Images are segmented with the unsupervised mean shift algorithm, which groups pixels into objects, using a trial-and-error approach for parameters. Objects are then classified with a linear-kernel SVM (regularization parameter 1).

The ForCM fusion. Rather than choosing between DL and OBIA, the pipeline takes heatmap weights from the ResUNet or Attention UNet prediction map (red intensity indicating forest probability, blue the opposite) and combines them with spectral features extracted from the image objects. These combined features train the SVM, whose object-level outputs are thresholded into a final binary forest/non-forest map. The same mean shift and SVM settings are used for both plain OBIA and the fused versions, so the comparison is like-for-like.

Why This Matters

Impact on research. The work positions DL-OBIA fusion as a viable alternative to using either technique alone for forest mapping, and it provides a like-for-like benchmark (same images, same segmentation parameters, same classifier) against traditional OBIA. Earlier DL-OBIA combinations had been applied to tasks such as check dam extraction and rock glacier detection; this paper argues forest cover mapping from multispectral Sentinel-2 imagery had not been tested this way before.

Real-world applications:

• National and regional forest monitoring programs that need accurate forest/non-forest baselines and change detection.
• Deforestation rate estimation and carbon stock accounting, which the authors highlight as a future use of the method.
• Conservation planning and biodiversity protection in tropical regions such as the Amazon, where canopy overlap and occlusion make boundaries hard to resolve.
• Land cover and land use assessment in resource-constrained settings, since the pipeline relies on free satellite data and free software.

Industry relevance. The method matters to geospatial analytics firms, environmental consultancies, government forestry and environment agencies, and NGOs that already use QGIS. Because it avoids commercial GIS licenses, it lowers the barrier to entry for organizations with limited budgets, while still delivering accuracy in the low-to-mid 90% range.

Future Directions

1. Temporal monitoring: Applying ForCM to consecutive years of imagery to track forest cover change and quantify the rate of deforestation and other environmental transformations.
2. Better segmentation: Testing multi-resolution segmentation, which the authors note is available in paid GIS software and may outperform the mean shift approach used here, alongside systematic optimization of segmentation parameters.
3. Larger and more diverse data: Experimenting with more complex, larger, more diverse, and higher-quality image datasets to test how well the method generalizes beyond the Amazon Rainforest.
4. Robustness and reproducibility: Reducing dependence on manual trial-and-error parameter tuning and unclear low-quality inputs, which the paper identifies as the main sources of variability in its own results.

Target Audience

Remote sensing and GIS researchers, machine learning practitioners working on semantic segmentation for Earth observation, environmental scientists and forest monitoring agencies, and graduate students interested in how deep learning can be layered onto established geospatial workflows. Readers with basic familiarity with segmentation models and GIS software will get the most from the architectural comparisons and the accuracy tables, while those focused on operational forestry will value the emphasis on free, open-source tooling.

Authors’ abstract

This research proposes "ForCM", a novel approach to forest cover mapping that combines Object-Based Image Analysis (OBIA) with Deep Learning (DL) using multispectral Sentinel-2 imagery. The study explores several DL models, including UNet, UNet++, ResUNet, AttentionUNet, and ResNet50-Segnet, applied to high-resolution Sentinel-2 Level 2A satellite images of the Amazon Rainforest. The datasets comprise three collections: two sets of three-band imagery and one set of four-band imagery. After evaluation, the most effective DL models are individually integrated with the OBIA technique to enhance mapping accuracy. The originality of this work lies in evaluating different deep learning models combined with OBIA and comparing them with traditional OBIA methods. The results show that the proposed ForCM method improves forest cover mapping, achieving overall accuracies of 94.54 percent with ResUNet-OBIA and 95.64 percent with AttentionUNet-OBIA, compared to 92.91 percent using traditional OBIA. This research also demonstrates the potential of free and user-friendly tools such as QGIS for accurate mapping within their limitations, supporting global environmental monitoring and conservation efforts.