From Machine Learning to Large-Scale EO Products: Best Practices for Making Maps

AI summary

Overview

Research area: Earth observation (EO) and machine learning — specifically the end-to-end engineering and scientific practice of turning satellite data into large-scale geospatial map products.

Technical level: Intermediate. The paper assumes familiarity with remote sensing terminology (radiance, reflectance, swaths, tiles) and standard ML practice (data splits, uncertainty, validation), but it is written as a practical guide rather than a technical derivation.

Scope: A condensed, end-to-end best-practices guide covering six interconnected stages of the EO mapping pipeline, from satellite data access through preprocessing, model training, uncertainty quantification, map production, and independent map validation.

What This Paper Is About

Advances in machine learning and large computing infrastructure have sharply lowered the barrier to producing global-scale maps from Earth observation data, but the field has not converged on established best practices. Design decisions made early in a pipeline — which data provider to use, how to preprocess, how to split training data, how to quantify uncertainty — propagate quietly into the final product, and much of the practical knowledge needed to avoid these traps is dispersed across application domains. The paper's goal is to consolidate that dispersed knowledge into a single, concise, stage-by-stage account of recommended practices, recurring pitfalls, and open gaps.

Key Contributions

1. A unified, end-to-end framing of the EO mapping pipeline. The paper organizes the discussion around six interconnected themes — the EO data infrastructure landscape, data selection and preprocessing, ML dataset construction and model training, uncertainty quantification, map production and distribution, and validation — and emphasizes the coupling between them rather than treating stages in isolation.

2. A provider- and platform-level comparison of the EO data access landscape. The appendix catalogues major data providers (ESA CDSE, USGS EarthExplorer, NASA Earthdata, and re-distribution platforms such as Google Earth Engine, Microsoft Planetary Computer, and AWS S3) along dimensions of server-side operations, feasibility of bulk archive download, and availability of collocated compute.

3. A distinction between model validation and map validation. The paper argues explicitly that held-out test metrics from the same data collection process measure generalization within that distribution, not the accuracy of the final map across its full extent, and that independent, probability-sampled reference data with a known sampling design is required for the latter.

4. A practical treatment of under-appreciated pitfalls. These include the spatial autocorrelation that inflates evaluation metrics, the need for blocked spatial cross-validation, the effect of grid choice on sampling and area estimation, patch-border artifacts at inference, and the fact that no existing uncertainty quantification method is simultaneously cheap, scalable, and well-calibrated.

The paper is a condensed version of a longer online guide at ghjuliasialelli.github.io/ML-EO-Maps/.

Main Findings

• Data access is not neutral. Different providers expose different processing levels and retrieval capabilities. In its COPERNICUS/S2_HARMONIZED collection, Google Earth Engine automatically corrects the radiometric offset introduced for Sentinel-2 data after Processing Baseline 04.00, and applies a full preprocessing chain to Sentinel-1 GRD data (thermal noise removal, data calibration, multi-looking, and range-doppler terrain correction) rather than redistributing the GRD product as given by ESA. The paper notes this can cause distribution shifts when a model trained on one provider's data is deployed on another's.

• Data gravity constrains the whole pipeline. Modern EO archives have grown to a scale where even freely available data is often impractical to move, making the position of compute relative to data a primary design factor. The paper splits this into a data-to-compute regime (limited by network bandwidth, provider rate limits, and egress costs) and a compute-to-data regime (compute provisioned in the same cloud region as the data), noting that spot instances are cheaper but require checkpoint-and-resume pipelines to handle preemption.

• Preprocessing cost compounds at inference scale. Whatever pipeline is used for training must be reproduced identically over the full mapped domain, so every additional processing step and temporal input adds operational cost. Aggressive filtering is acceptable when curating a training dataset, but inference must still produce predictions over the entire deployment area, including in persistently difficult conditions.

• Sensor and product artifacts require explicit handling. Sentinel-2 products exhibit artifacts from incomplete orbits and split partial products, where a tile that should be fully covered by one orbit yields two products sharing the same date, orbit, and footprint. Landsat 7 scenes acquired after the 2003 scan-line corrector failure exhibit systematic data gaps. If unhandled, these can cause pixel duplication, bias dataset statistics, and produce visible seams or gaps in the final map.

• Reference data coverage is uneven worldwide. OpenStreetMap, for example, offers more coverage in the Global North than the Global South, reflecting both disparities in accessibility and the priorities of the communities that create and curate these datasets. Models trained without accounting for this imbalance risk learning a skewed representation of the Earth's surface.

• Temporal strategy is a cost decision. Four broad options are contrasted: single time-steps (most cost-efficient but sacrifice temporal information), time-series (richest, capturing seasonal variation, but memory-intensive), composites (a pragmatic middle ground distilling a period into a single time-step), and time series of composites. The choice directly affects the volume of data that must be downloaded or streamed at inference time, which at global scale can be the binding constraint.

• Spatial autocorrelation demands blocked splits. Random splits leave train and test points autocorrelated, producing optimistic evaluation. Spatial K-fold block cross-validation divides the area into non-overlapping blocks per fold; spatial buffers or inter-fold distances exceeding the autocorrelation range of the target variable avoid cross-fold contamination.

• Grid choice affects sampling and evaluation validity. Candidate grids — Plate carrée, Web Mercator, UTM/MGRS, Equi7 Grid, Discrete Global Grid Systems such as S2 Geometry, H3 and HEALPix, Major TOM, and the Equal Earth grid — differ in whether they are equal-area, conformal, and globally continuous, and no single grid satisfies all three desiderata. The paper advocates decoupling the indexing strategy from the patch projection: indexing should use a global grid that minimizes over- and under-sampling and overlap, while the patch projection only needs to minimize distortion at the scale of a single patch.

• Architecture choice is an operational decision. Vision Transformers and foundation models are increasingly explored for EO tasks, but convolutional encoder-decoders remain the workhorse of operational pipelines because their higher throughput makes wall-to-wall inference tractable.

• Data augmentation can absorb preprocessing burden. Rather than explicitly correcting sensor artifacts or missing modalities, one can simulate them during training. Examples include adding noise to Sentinel-2 surface reflectance values for spectral robustness, dropping timesteps to simulate cloud-induced gaps or missing modalities, and simulating narrow no-data seams in Sentinel-1 GRD mosaics as well as swath boundaries, misregistration, and detector failures.

• Uncertainty comes from more than the model. Total uncertainty arises from upstream (input) uncertainty (sensor noise, atmospheric correction errors, cloud mask failures), label uncertainty (measurement error in reference data), and model (epistemic) uncertainty. Many UQ methods address only model uncertainty, and the paper recommends making that gap explicit.

• No UQ method is cheap, scalable, and well-calibrated at once. Gaussian Processes are calibrated if the kernel is correct but do not scale; deep ensembles scale but are expensive and can be miscalibrated for spatial data because members trained on the same spatially autocorrelated data agree where they are collectively wrong; heteroskedastic regression is cheap but blind to epistemic uncertainty; conformal prediction gives distribution-free coverage as a cheap post-hoc wrapper but only marginally; conformalized quantile regression improves on this for classification but coverage still holds on average over the test distribution, not for any specific subset.

• Naive aggregation understates uncertainty at regional scales. If pixel errors were independent, aggregate uncertainty would shrink as 1/√n with the number of pixels, but positively correlated errors do not cancel upon averaging. The paper cites Wadoux et al. showing that several high-profile studies reported aggregate uncertainty without accounting for spatial autocorrelation, leading to substantially underestimated confidence intervals, and Johnson et al. finding that residual variance from spatial correlation dominated all other sources of uncertainty for biomass aggregated to ownership parcels.

• Patch-border artifacts are a known, addressable failure mode. Predictions degrade toward patch borders due to zero-padding, non-unary strides, and windowed attention. Two strategies address this: overlapping patches with weighted blending using distance-based weights, and padding and cropping where each patch is expanded with context and only center predictions are retained. The paper notes that blending smooths transitions but blurs predictions in the overlap zone and complicates uncertainty propagation, whereas padding preserves model output and keeps uncertainty maps faithful but leaves residual artifacts if the receptive field is smaller than the padding.

• Map validation is distinct from model validation. A held-out test set from the same collection process shares its biases and coverage gaps; uncertainty quantification methods have limitations; and comparison against another map only measures inter-product consistency. None are substitutes for independent validation. The gold standard is a probability-based sampling design where every location has a known, non-zero chance of selection, with stratified random sampling, area-weighted confusion matrices, per-class accuracy reported from both the user's (precision) and producer's (recall) perspective, and confidence intervals. For continuous-valued products such as biomass, strict probability sampling may not be practical, and alternative validation should state which ecoregions, land cover types, and target variable ranges are underrepresented.

Note on quantitative results: This is a best-practices guide and reports no experimental results, benchmark scores, dataset sizes, or model performance numbers. The specific figures it does contain are technical parameter values, described in the Methodology section below.

Methodology in Plain English

This paper is not an experimental study; it is a practitioner-oriented synthesis. The authors describe it as the result of a broad effort to engage with practitioners across the EO and ML communities, distilling common practices, recurring pitfalls, and recommended approaches into a concise account.

The content is organized as a walkthrough of the pipeline in the order a practitioner encounters it. It begins with the data access landscape, enumerating providers and platforms and their trade-offs. It then moves to data selection and preprocessing, covering how individual observations differ in usability, how training samples should be distributed geographically and temporally, and which corrections are typically needed per sensor. Next comes ML dataset construction and model training: storage formats, spatial splitting strategies, grid choices, architecture selection, augmentation as an alternative to explicit correction, and the distinction between held-out model validation and downstream map validation.

Uncertainty quantification is treated as its own stage, covering sources of uncertainty, the trade-offs among methods, calibration diagnostics such as coverage and reliability plots, standardized residuals (z-scores with ideal mean 0 and standard deviation 1), and proper scoring rules including the interval score and CRPS. Map production and distribution covers compute environments, artifact mitigation, and sharing conventions such as FAIR principles, Cloud Optimized GeoTIFFs, Zarr, and SpatioTemporal Asset Catalog (STAC) indexing. The final stage covers map validation through probability sampling and design-based assessment.

Supporting technical detail is placed in appendices: a table of data providers, Sentinel-2 artifact cases, the Sentinel-2 radiometric offset formulas, a comparison of AI4EO storage formats, illustrations of spatial splitting strategies and patch-artifact mitigation, and a table of candidate grids.

A representative worked example of the paper's detailed technical guidance is its treatment of the Sentinel-2 radiometric offset. Since the deployment of Sentinel-2 Processing Baseline (PB) 04.00 in January 2022, a radiometric offset is added to the digital number values of all bands. For L1C data the conversion to top-of-atmosphere reflectance is (DN + RADIO_ADD_OFFSET) / QUANTIFICATION_VALUE, where the offset is −1000 for all bands for PB ≥ 04.00 and 0 otherwise, and QUANTIFICATION_VALUE = 10,000. For L2A data the analogous formula uses BOA_ADD_OFFSET and BOA_QUANTIFICATION_VALUE = 10,000. The paper stresses that the offset applies to any product with Processing Baseline ≥ 04.00, which should be determined from the PROCESSING_BASELINE field in the metadata rather than the acquisition date, because ESA reprocessed the entire historical archive as part of the Collection-1 reprocessing campaign completed in 2024, so even early acquisitions now carry the offset convention. It also warns that some third-party platforms such as Google Earth Engine's harmonized collections apply the offset upstream, in which case it must not be applied a second time.

Why This Matters

Impact on research. The paper argues that several common evaluation practices are systematically optimistic. Spatial autocorrelation inflates metrics when splits are random; held-out test sets drawn from the same collection process as training data share its biases and coverage gaps; and aggregate uncertainty reported without accounting for spatially correlated errors can produce substantially underestimated confidence intervals. By naming these as coupled pipeline problems rather than isolated methodological quibbles, the paper provides a common vocabulary for a research community that has grown rapidly and drawn in practitioners from both EO and ML who may not share the same tacit knowledge.

Real-world applications:

• Climate reporting and carbon markets. The paper notes that international frameworks for climate reporting require quantified uncertainty, and that carbon credit markets depend on it. Uncertainty is described as what transforms a map from a static product into a scientifically credible tool that guides action.

• Global land cover and vegetation mapping. Maps of human settlement, vegetation structure, and natural hazards are cited as examples of products now routinely generated at resolutions that would have been infeasible a few years ago.

• Disaster and natural hazard response. The scientific framing covers natural hazards among the thematic areas these products span, where inference must cover the entire deployment area even under persistently difficult conditions.

• National and regional land monitoring. Probability-based sampling with area-weighted confusion matrices and per-class precision and recall gives agencies defensible accuracy estimates over a full mapped domain rather than only at conveniently available locations.

Industry relevance. The paper frames architecture selection as an operational decision, because every forward pass during training is repeated billions of times at inference, and it frames compute placement as a cost question shaped by data gravity, network bandwidth, egress fees, provider rate limits, and spot-instance preemption. These are the concerns of organizations running production pipelines rather than publishing research prototypes. The recommendations on FAIR-compliant distribution, Cloud Optimized GeoTIFFs, Zarr, and STAC indexing speak directly to teams building data products that others must discover, download, and use.

Future Directions

• Closing the aggregate uncertainty gap. The paper identifies the handling of spatially correlated errors in aggregate uncertainty as "one of the largest gaps between current practice and what is needed for rigorous area-based reporting," and recommends as a practical minimum characterizing residual spatial correlation in a small number of ecologically distinct validation regions. Scaling this to global products remains open.

• Developing UQ methods that are cheap, scalable, and well-calibrated simultaneously. The paper states plainly that no existing method achieves all three, leaving room for methods whose coverage guarantees hold for specific subsets of the test distribution rather than only on average.

• Establishing validation practice for continuous-valued products. For maps like biomass, strict probability sampling may not be practical, and the paper calls for methods that characterize which ecoregions, land cover types, and variable ranges are underrepresented in reference data, while stating limitations explicitly rather than ignoring them.

• Converging on preprocessing and distribution conventions across providers. The distribution shift risk illustrated by differing platform-level preprocessing of Sentinel-1 GRD and Sentinel-2 data suggests a need for greater consistency, or at least for documentation that lets practitioners reason about the differences. The paper also frames its contribution as an invitation for continued exchange across the EO and ML communities, and points readers to the longer online guide for additional depth.

Target Audience

This paper benefits practitioners who are building or reviewing large-scale Earth observation map products: remote sensing scientists and ML engineers moving from prototype models to operational pipelines; researchers who need to report accuracy and uncertainty defensibly; geospatial data product teams concerned with compute placement, storage formats, and distribution; and reviewers or downstream users who need to judge whether a published map's accuracy assessment is credible. It is also useful for ML researchers entering the EO domain from standard computer vision, since several of the pitfalls it describes — spatial autocorrelation, data gravity, provider-dependent preprocessing, and the model-validation versus map-validation distinction — have no direct analogue in conventional vision benchmarks. Given the paper positions itself as a condensed version of a longer guide, readers seeking implementation-level depth are directed to the online version at ghjuliasialelli.github.io/ML-EO-Maps/.

Authors’ abstract

Recent years have seen a rapid expansion in the production of large-scale geospatial maps derived from Earth observation (EO) data, driven largely by advances in machine learning (ML) and large computing infrastructure. Although the barrier to generating such maps has dropped substantially, established best practices have yet to emerge, and design decisions made early in the pipeline can quietly propagate errors into the final product. Producing a technically sound and scientifically credible product remains challenging. Choices made at every stage are tightly coupled: preprocessing decisions shape the training signal, dataset design governs what the model can learn and how reliably its performance can be assessed, and global-scale inference introduces engineering challenges in compute and data access at scale, as well as artifact mitigation. Furthermore, uncertainty quantification and independent map validation each require dedicated methodological attention that is often underestimated. This paper presents a concise, end-to-end account of the recommended practices spanning the pipeline from satellite data to an operational map product. We organize the discussion around six interconnected themes: the EO data infrastructure landscape, data selection and preprocessing, ML dataset construction and model training, uncertainty quantification, map production and distribution, and validation. This paper is a condensed version of a longer guide that provides greater depth across all stages, accessible online at ghjuliasialelli.github.io/MLEO-Maps/.