MMLSv2: A Multimodal Dataset for Martian Landslide Detection in Remote Sensing Imagery

AI summary

Overview

Research area: Computer vision / remote sensing — multimodal semantic segmentation, planetary geomorphology (Martian landslide mapping).

Technical level: Intermediate. Familiarity with semantic segmentation architectures (U-Net, DeepLab, SegFormer), IoU-style metrics, and basic geospatial raster concepts helps, but the dataset description itself is accessible.

Scope: The paper introduces MMLSv2, a seven-band Martian surface imagery dataset for binary landslide segmentation, together with a geographically isolated test set and benchmark results from six segmentation architectures.

What This Paper Is About

Landslide mapping from satellite or orbital imagery is normally done by hand, which is slow and hard to scale, and deep learning could automate it — but landslide segmentation models are bottlenecked by a lack of large, well-annotated, spatially unbiased datasets, especially for planetary surfaces. This paper presents MMLSv2, a revised multimodal dataset of Martian orbital imagery with seven aligned data channels and pixel-level landslide masks, aimed at binary landslide segmentation. The goal is to provide a cleaner, better-partitioned benchmark that also tests whether a model can generalize to a region it has never seen.

Key Contributions

1. A multimodal dataset with seven data channels. MMLSv2 provides spatially aligned multimodal observations combining red, green, and blue spectral bands, digital elevation model, slope, thermal inertia, and grayscale representations for binary landslide segmentation from orbital imagery.

2. A fully isolated test set for spatial generalization. A set of 276 images from a geographically disjoint region (separated from the base region by a fixed east-west boundary at the raster level) is released and kept entirely out of training and validation, so models are evaluated on an unseen area.

3. A refined version of the earlier MMLSv1 dataset. MMLSv2 corrects and refines pixel-level annotations, adds a dedicated partitioning strategy, and includes more than 100 additional images. The revision adds previously omitted landslides (including events without well-defined depositional zones) and removes ambiguous complex depositional regions from existing annotations to reduce label noise and spurious correlations.

4. Benchmark experiments across multiple architectures. Six segmentation models are trained and evaluated under a unified configuration, with performance reported on both the standard test split and the isolated test set, plus an ablation on which input band combinations help.

Main Findings

• Dataset composition: MMLSv2 totals 664 images across its base splits — 465 training, 66 validation, and 133 test images — plus a separate isolated test set of 276 images. All patches are 128 × 128 pixels extracted on a deterministic grid with zero overlap.

• Foreground statistics are comparable across base splits but not in the isolated set: Average landslide foreground ratio is 35.41% (train), 31.53% (val), 33.82% (test), and 21.83% (isolated test), with standard deviations of 25.64%, 24.05%, 25.05%, and 17.08% respectively. Minimum–maximum foreground ratios span 0.02–99.52% (train), 0.08–90.32% (val), 0.10–90.67% (test), and 0.01–71.95% (isolated test). The lower average and reduced dispersion in the isolated set indicate it contains a larger proportion of sparse landslide events, making it structurally different from training data.

• Stable training with headroom remaining: On the standard test split, all evaluated models converge properly with mean IoU values in the 0.81–0.83 range. The highest mIoU is DeepLabV3+ at 0.829, followed by PSPNet at 0.828, U-Net++ at 0.823, DeepLabV3 at 0.817, U-Net at 0.814, and SegFormer at 0.812.

• Foreground class is the harder class: Class-wise analysis shows most performance degradation occurs in the foreground (landslide) class rather than background, indicating the task remains non-trivial.

• Efficiency trade-offs: Training time follows architectural complexity. PSPNet is the fastest to train at 0.027 hours, followed by the U-Net variants (U-Net++ 0.085 h, U-Net 0.088 h, DeepLabV3+ 0.048 h, DeepLabV3 0.063 h). SegFormer has the highest training cost at 0.131 hours and the highest inference latency (0.041 s per image) without a corresponding gain in mIoU. Convolutional models cluster in a more favorable accuracy-efficiency region.

• Performance drops on the isolated test set: All models show a consistent mIoU decrease from above 0.80 to the 0.70–0.73 range. The best isolated-set mIoU is SegFormer at 0.735, followed by DeepLabV3 at 0.725, U-Net at 0.721, PSPNet at 0.709, U-Net++ at 0.708, and DeepLabV3+ at 0.696. Foreground IoU falls from values above 0.70 to slightly above 0.60 in the best-performing model (SegFormer, at 0.614), while background performance remains stable.

• Additional bands improve segmentation: Using U-Net, mIoU rises consistently from 0.711 with RGB alone, to 0.762 with RGB-DEM, 0.805 with RGB-DEM-Slope, 0.812 with RGB-DEM-Slope-Thermal, and 0.814 with RGB-DEM-Slope-Thermal-Gray. DEM and thermal cues steadily enhance foreground delineation. Inference latency stays at 0.005 s across all configurations, while training time increases only marginally (from 0.082 h with RGB to 0.088 h with all five listed configurations).

• Failure patterns are consistent across models: All architectures recover large, continuous landslide regions well, but show partial detections, boundary inaccuracies, or missed regions on fragmented, thin, discontinuous, elongated, or weakly contrasted landslides — a pattern that is worse on the isolated test set.

Methodology in Plain English

The researchers focused on the Valles Marineris region of Mars, chosen because its steep escarpments, complex terrain, and pervasive mass-wasting processes make it challenging and representative for landslide analysis. They combined several remote sensing sources: nighttime infrared imagery from THEMIS aboard Mars Odyssey, a global thermal inertia mosaic at 100 m resolution, Context Camera (CTX) imagery from the Mars Reconnaissance Orbiter at approximately 6 m resolution, Digital Elevation Models from MOLA aboard Mars Global Surveyor blended with HRSC data at 200 m resolution, and the Viking colorized global mosaic at 232 m. Because these sources come from different missions with different spatial resolutions and native grids, they co-registered and processed everything in ESRI ArcGIS. Slope was computed from the MOLA DEM using the ArcGIS Slope tool.

Landslides were manually identified and digitized as polygons following established morphological criteria, and study regions were labeled as landslide or non-landslide. The result is a seven-band composite per tile. To make masks line up exactly, the researchers forced the mask GeoTIFF to share the same coordinate reference system, affine transform, width, and height as the reference composite, resampling with nearest-neighbor interpolation to preserve categorical labels, then binarized the result so landslide pixels are foreground (1) and everything else — including nodata — is background (0).

For evaluation of generalization, they split the aligned mosaics into two non-overlapping subregions with a fixed east-west boundary. Patches from one subregion form the isolated test set and never enter training or validation. Both regions are tiled into 128 × 128 pixel patches on a deterministic grid with zero overlap, with an extra terminal origin to cover edges without padding.

For partitioning the base region, they avoided random assignment of patches, because neighboring patches in continuous terrain share highly correlated visual patterns — splitting them randomly leaks spatial information and lets models exploit context related to training data. They also noted that random splits don't preserve the uneven distribution of landslide content. Instead, they combined foreground-based stratification with spatial grouping. Each patch's foreground ratio (the proportion of landslide pixels in its mask) serves as a proxy for landslide dominance and sample difficulty. Patches are grouped into 2 × 2 spatial blocks using their grid indices, and every patch in a block goes to the same split, so adjacent patches cannot land in different subsets. Each block gets a representative foreground ratio (the average over its patches), and blocks are stratified using quantile-based binning so landslide dominance is consistently represented across subsets.

Benchmarks covered U-Net, U-Net++, PSPNet, DeepLabV3, DeepLabV3+, and SegFormer, all trained under one unified configuration: Adam optimizer, learning rate 0.001, cross-entropy loss, a step-based scheduler reducing the rate by a factor of 0.1 every 30 epochs, 100 epochs, batch size 128, on a single NVIDIA A100 GPU with 40 GB memory, using all seven input bands and basic augmentation via random flips and rotations. Metrics were precision, recall, F1 score, background IoU, foreground IoU, and mean IoU, plus inference and total training time. For each architecture, the checkpoint with the highest validation mIoU was selected and evaluated on both test splits.

Why This Matters

The paper targets a real bottleneck in applied landslide research: deep learning models need large, well-annotated data, but landslides are spatially sparse, so class imbalance is severe, and labels are hard to obtain in regions with limited satellite coverage or expert annotation. By publishing a curated multimodal benchmark with a deliberately disjoint test region, the work gives researchers a way to measure generalization rather than memorization — something random patch-level splits cannot do in spatially continuous terrain.

Real-world applications:

• Disaster risk mitigation and emergency response: Faster, automated landslide delineation from orbital imagery supports response planning and hazard inventories, the use cases the paper cites for landslide identification.
• Land-use planning and infrastructure safety: Landslide inventories inform where construction, roads, and settlements are viable in steep terrain.
• Planetary geomorphology and mission science: Martian landslide mapping feeds into understanding surface processes and mass wasting on a body the paper describes as one of the most Earth-analogous in terms of surface processes and geomorphological dynamics.
• Benchmarking general-purpose segmentation models under domain shift: The isolated test set provides a controlled setting for testing robustness of segmentation architectures under out-of-distribution conditions.

Industry relevance: teams building geospatial AI for hazard monitoring, insurance and reinsurance risk assessment, mining and civil engineering site evaluation, and satellite imagery analytics vendors can use the dataset and the reported accuracy-efficiency trade-offs to choose models. The finding that convolutional models reach comparable mIoU to a Transformer at substantially lower training and inference cost (SegFormer at 0.131 h training and 0.041 s inference versus PSPNet at 0.027 h and 0.005 s, with mIoU 0.812 versus 0.828) is directly relevant to deployed systems where latency and compute budgets matter.

Future Directions

• Architecture-specific tuning: The paper states the unified configuration was intentionally simple and reports results as reference baselines rather than performance upper bounds, leaving room for specialized training strategies.
• Better multimodal fusion: Current experiments integrate modalities by direct band concatenation. The authors suggest multi-branch architectures or modality-aware fusion could better exploit the complementary spectral and auxiliary inputs.
• Broader model coverage: Evaluation was limited to a subset of widely used architectures; extending to additional model families could further characterize the challenges and potential of MMLSv2.
• Closing the generalization gap: The consistent mIoU drop from above 0.80 to 0.70–0.73 on the isolated set, driven mainly by foreground IoU falling from above 0.70 to slightly above 0.60, leaves open the question of how to make models robust to fragmented, elongated, small-scale, and structurally novel landslide patterns.

Target Audience

Researchers and practitioners in remote sensing, computer vision, and planetary geoscience who work on semantic segmentation or landslide hazard mapping. It is also useful for machine learning engineers needing a multimodal benchmark with an explicit out-of-distribution test set, and for planetary scientists studying mass-wasting processes in Valles Marineris. Readers without a segmentation background can still follow the dataset design and partitioning rationale, though interpreting the benchmark tables requires familiarity with IoU, precision, recall, and F1 metrics.

Authors’ abstract

We present MMLSv2, a dataset for landslide segmentation on Martian surfaces. MMLSv2 consists of multimodal imagery with seven bands: RGB, digital elevation model, slope, thermal inertia, and grayscale channels. MMLSv2 comprises 664 images distributed across training, validation, and test splits. In addition, an isolated test set of 276 images from a geographically disjoint region from the base dataset is released to evaluate spatial generalization. Experiments conducted with multiple segmentation models show that the dataset supports stable training and achieves competitive performance, while still posing challenges in fragmented, elongated, and small-scale landslide regions. Evaluation on the isolated test set leads to a noticeable performance drop, indicating increased difficulty and highlighting its value for assessing model robustness and generalization beyond standard in-distribution settings. Dataset will be available at: https://github.com/MAIN-Lab/MMLS_v2