Super-Resolved Canopy Height Mapping from Sentinel-2 Time Series Using Airborne LiDAR HD Reference Data across Metropolitan France

AI summary

Overview

Research area: Remote sensing and computer vision applied to forestry — specifically, canopy height estimation from satellite image time series, combined with single-image super-resolution.

Technical level: Advanced. The paper assumes familiarity with deep learning architectures (Residual Dense Networks, temporal self-attention transformers, sub-pixel convolution), remote sensing data products (Sentinel-2 Level 2A, airborne LiDAR point clouds), and standard geospatial regression metrics.

Scope: The paper introduces THREASURE-Net, an end-to-end deep learning framework that jointly performs canopy height regression and super-resolution from Sentinel-2 time series, trained on airborne LiDAR HD reference data across metropolitan France, and evaluated in three output resolutions (2.5 m, 5 m, 10 m).

What This Paper Is About

Mapping forest canopy height at fine spatial scales is important for carbon accounting, biodiversity monitoring, and forest health assessment, but the freely available Sentinel-2 satellite constellation only observes at 10 m resolution, while commercial very-high-resolution imagery is expensive to acquire at national scale. This paper asks whether a single deep learning model can simultaneously (a) predict canopy height from Sentinel-2 time series and (b) increase the effective spatial resolution of those predictions, using only free satellite data and LiDAR-derived height references — with no pretrained super-resolution network and no very-high-resolution optical imagery in the training loop.

Key Contributions

1. A fully end-to-end model with no pretrained components: THREASURE-Net does not rely on any pretrained super-resolution network and does not use any very-high-resolution reference optical imagery to train its super-resolution module; it learns solely from LiDAR-derived height information.

2. Sentinel-2-only accuracy competitive with VHR-based methods: The model is trained solely on Sentinel-2 data yet achieves accuracy comparable to models trained on very-high-resolution satellite data, while additionally recovering fine-scale spatial details that are not visible at the native 10 m resolution.

3. Seasonally conditioned predictions: During training, model predictions are conditioned on the LiDAR acquisition date, explicitly accounting for the seasonal variability of the reference data. This is implemented by replacing the temporal transformer's learnable queries with queries projected from a sinusoidal encoding of the LiDAR acquisition day-of-year.

4. Joint height regression and tree masking, robust to irregular inputs: The model simultaneously predicts canopy height and a tree/non-tree classification mask (whereas existing methods restrict height prediction to pre-defined forest areas using an external mask), and it handles irregular Sentinel-2 time series without explicitly requiring cloud-free observations.

Main Findings

• Reported mean absolute errors: THREASURE-Net achieves mean absolute errors of 2.63 m at 2.5 m resolution, 2.70 m at 5 m resolution, and 2.89 m at 10 m resolution.
• Outperforms Sentinel-based state of the art: The authors state that their approach outperforms existing state-of-the-art methods based on Sentinel data.
• Competitive with VHR-based methods: The approach is described as competitive with methods based on very-high-resolution imagery, despite using only freely available Sentinel-2 data.
• Coherent fine-scale detail: The super-resolved outputs contain coherent fine-scale spatial details that are not visible in predictions at the native resolution.
• Comparison baselines used: The paper compares against FORMS-T (10 m, Sentinel-1/Sentinel-2 with GEDI reference), Open-Canopy (1.5 m, single VHR SPOT-6/7 image with LiDAR HD), and the Global Canopy Height (GCH) model (10 m, GEDI reference with a single-date Sentinel-2 image), plus four independent ALS height maps from ONF (Carcans-Hourtins, Déodatie, Lajoux-Lafresse, Mouterhouse).
• Not reported in the available content: The truncated paper content does not report the numeric values for R², RMSE, rMAE, IoU, the Frequency Attenuation Profile (FAP) results, the independent ALS comparisons, the cross-comparison with FORMS-T / Open-Canopy / GCH, or the results of the ablation study on loss functions. These metrics are described as being computed, but their values do not appear in the provided text.

Methodology in Plain English

The reference data. The authors use France's national airborne LiDAR HD campaign (minimum density 10 pulses per m², patches acquired 2021–2024, campaign ongoing). Each 3D point cloud is projected onto a regular pixel grid at 10 m, 5 m, or 2.5 m, and the 95th percentile of elevation values within each pixel becomes the target height. Only pixels corresponding to vegetation taller than 1.5 m receive a height value; all others are set to zero. Because tall crops such as corn can be misclassified as forest vegetation, a crop mask from the French agricultural parcel register (RPG) is applied, and vegetation inside crop parcels below 5 m is excluded from the annotations.

The satellite inputs. Sentinel-2 Level 2A time series from 2018–2024, restricted to images acquired May through October of the same year as the reference LiDAR patch. Images with more than 50% cloud coverage or more than 10% missing data are filtered out. Ten spectral bands are used (B02, B03, B04, B05, B06, B07, B08, B8A, B11, B12) together with a cloud mask and acquisition angles (sun azimuth and zenith, satellite mean view azimuth and zenith), all resampled to 10 m. The angles are encoded with trigonometric transformations to respect angular periodicity.

The architecture. The model has four stages. First, a spatio-spectral feature extraction module based on Residual Dense Blocks (from the Residual Dense Network designed for natural-image single-image super-resolution) processes each image independently, using dense connections for feature reuse while remaining lighter than Residual-in-Residual Dense Blocks. Second, a Lightweight Temporal Self-Attention module aggregates information across the time series, using sinusoidal positional encoding to represent acquisition dates. Crucially, the learnable queries of the original attention formulation are replaced by queries derived from the LiDAR acquisition date, so the prediction is conditioned on when the reference data were captured; day-of-year values are normalized relative to January 1st for Sentinel-2 and July 1st for LiDAR, and the year is omitted to generalize to unseen years. Third, an upsampling block applies sub-pixel convolution (pixel shuffle), progressively doubling resolution by a factor of two until the target is reached — but it is applied after temporal encoding rather than before, to reduce memory consumption and to share a single temporally encoded feature map. Sub-pixel weights are initialized following the Aitken et al. approach within sub-pixel groups and small random noise is added to break symmetry and avoid checkerboard artifacts. Fourth, a multi-layer perceptron with two parallel heads produces a height map and a binary tree/non-tree mask; the classification head is optimized on detached features so its gradients do not affect the shared representation.

Training setup. Three loss terms are combined: an L1 (MAE) loss for height, binary cross-entropy for classification, and a Weighted Gradient Difference Loss (WGDL) to sharpen edges and textures. The loss weights are ω_height = 1, ω_classif = 1, and ω_wgdl = 0.33, with non-tree pixels weighted at w_non-tree = 0.1. The height loss is averaged within each patch before averaging across the batch, to avoid imbalance from homogeneous forest patches. In the WGDL, pixel weights are derived from normalized target gradient magnitudes, clamped to a minimum of λ_min, so that homogeneous regions are not excluded. The training data comprise 179,945 patches, with 19,133 for validation and 35,639 for testing; 15% of LiDAR HD tiles were reserved for testing, all adjacent tiles (including diagonal neighbours) were excluded as buffers, and the remainder was split 90/10 between training and validation. Input patches are 64×64 pixels (640×640 m²) with 250 m border margins to avoid convolution border effects. The maximum time-series length is 12, series shorter than 5 observations are discarded, and the network was trained on the CNES TREX HPC cluster with NVIDIA A100 and H100 GPUs in PyTorch 2.5.1 with CUDA 11.8 using mixed precision and the Adam optimizer with a cosine annealing warm-restarts scheduler.

Evaluation. Four regression metrics (R², MAE, RMSE, rMAE) plus IoU for the classification mask. Because super-resolved predictions can be misaligned with higher-resolution reference data, shifted evaluation is applied to the 5 m and 2.5 m models: several spatial offsets are tested and the alignment yielding the lowest MAE is selected, with the remaining metrics computed at that shift. Super-resolution quality is assessed with the Frequency Attenuation Profile (FAP), a Discrete Fourier Transform-based measure comparing attenuation curves of the HR reference, the SR prediction, and a bicubic-upsampled low-resolution input.

Why This Matters

Impact on research. The paper shows that multi-image super-resolution can be fused with a downstream prediction task in an end-to-end framework, and that a super-resolution module can be supervised purely by LiDAR-derived height information rather than by paired low-/high-resolution optical imagery. This sidesteps the computational burden and co-registration fragility of first super-resolving Sentinel-2 time series and then feeding them into a separate predictor. It also demonstrates that LiDAR acquisition timing can be injected as a conditioning signal, addressing a real source of label noise in remote sensing datasets where reference data are collected on a single date.

Real-world applications:

• National-scale forest carbon and biomass monitoring using only free satellite data.
• Detection of forest degradation, selective logging, and regeneration at fine spatial scales.
• Complementing national forest inventories — the paper notes that French forest monitoring still relies largely on field measurements and photointerpretation of high-resolution imagery, in contrast to Scandinavian countries.
• Annual canopy-height map production across large territories, enabled by the temporal continuity of Sentinel-2.

Industry relevance. The method targets the cost barrier that makes VHR commercial imagery (PlanetScope, Maxar, SPOT-6/7) impractical for national-scale model training. For forestry companies, carbon-credit verification bodies, and environmental agencies, a freely-deployable pipeline that yields 2.5 m canopy height maps could replace or augment expensive airborne survey cycles. The authors release source code at https://github.com/Global-Earth-Observation/threasure-net.

Future Directions

1. Temporal canopy height mapping. LiDAR HD observes each French area typically only once, which prevents temporal analysis using LiDAR HD alone. Extending THREASURE-Net from annual height maps to reliable change detection and disturbance tracking across years remains an open challenge.

2. Geographic generalization. The model was developed for metropolitan France, chosen for its climatic diversity (five distinct climates and 190 tree species across 91 sylvoécorégions) as a representative example of temperate European forests. Whether the approach transfers to tropical, boreal, or other forest types is not established in the available content.

3. Pushing super-resolution further. The upsampling factor is constrained to powers of two, and the paper notes that enlarging the RDB configuration for the 2.5 m model did not yield performance improvements. How much finer the resolution can go, and what architectural changes would be needed, is an open question.

4. Rigorous ablation and independent validation at scale. The paper plans an ablation study on patch-wise averaging, the WGDL loss, and the non-tree pixel weight, plus comparison against four independent ONF ALS height maps and three competing products (FORMS-T, Open-Canopy, GCH). The numeric outcomes of these analyses are not present in the available content and would strengthen claims about which components drive the reported accuracy.

Target Audience

This paper is most valuable to remote sensing and computer vision researchers working on super-resolution, multi-temporal satellite fusion, and forest structure estimation; to geospatial engineers building national-scale canopy height or biomass products; and to forestry agencies, carbon monitoring organizations, and environmental policy teams evaluating whether free Sentinel-2 data can substitute for commercial very-high-resolution imagery. Readers without a deep learning background will find the architectural details demanding, but the framing of the cost/accuracy trade-off and the reported error levels are accessible to applied forestry and GIS professionals.

Authors’ abstract

Fine-scale forest monitoring is essential for understanding canopy structure and its dynamics, which are key indicators of carbon stocks, biodiversity, and forest health. Deep learning is particularly effective for this task, as it integrates spectral, temporal, and spatial signals that jointly reflect the canopy structure. To address this need, we introduce THREASURE-Net, a novel end-to-end framework for Tree Height Regression And Super-Resolution. The model is trained on Sentinel-2 time series using reference height metrics derived from LiDAR HD data at multiple spatial resolutions over Metropolitan France to produce annual height maps. We evaluate three model variants, producing tree-height predictions at 2.5 m, 5 m, and 10 m resolution. THREASURE-Net does not rely on any pretrained model nor on reference very high resolution optical imagery to train its super-resolution module; instead, it learns solely from LiDAR-derived height information. Our approach outperforms existing state-of-the-art methods based on Sentinel data and is competitive with methods based on very high resolution imagery. It can be deployed to generate high-precision annual canopy-height maps, achieving mean absolute errors of 2.63 m, 2.70 m, and 2.88 m at 2.5 m, 5 m, and 10 m resolution, respectively. These results highlight the potential of THREASURE-Net for scalable and cost-effective structural monitoring of temperate forests using only freely available satellite data. The source code for THREASURE-Net is available at: https://github.com/Global-Earth-Observation/threasure-net.