Generalizable Slum Detection from Satellite Imagery with Mixture-of-Experts

AI summary

Overview

• Research area: Computer vision / remote sensing for urban poverty measurement — specifically, automatic segmentation of informal settlements ("slums") in very-high-resolution satellite imagery under cross-region domain shift.
• Technical level: Advanced. The paper assumes familiarity with semantic segmentation, transformer backbones, Mixture-of-Experts (MoE) routing, and test-time adaptation (TTA).
• Scope: The paper introduces GRAM (Generalized Region-Aware Mixture-of-Experts), a two-phase source-training plus test-time-adaptation framework for label-free slum segmentation across unseen cities, evaluated on three held-out African cities.

What This Paper Is About

Slums look very different from one part of the world to another — different roof materials, building shapes, density, and street layouts — so a segmentation model trained on labeled imagery from one set of cities tends to fail in cities it has never seen. Because pixel-level slum annotations are expensive and scarce, the authors ask whether a model can be adapted at test time to a new city using only unlabeled satellite imagery. Their answer is GRAM, which pairs a Mixture-of-Experts segmentation model trained on 12 cities across four continents with a pseudo-label filtering scheme based on agreement among region-specific experts.

Key Contributions

1. A million-scale, multi-continent slum segmentation dataset. The authors assemble labeled and pseudo-labeled imagery from 12 source cities across four continents (Africa, Asia, South America, Central America) plus three held-out African test cities, and release it publicly.
2. GRAM, a two-phase test-time adaptation framework. Source training integrates lightweight MoE blocks with region-specific gating into a transformer encoder; target adaptation fine-tunes the model on unlabeled target imagery using pseudo-labels, with no target ground truth used at any point.
3. A cross-region prediction consistency filter. A "stability score" measures the mean IoU between the pseudo-label from the classifier-selected expert and the pseudo-masks produced by all other region routings; only the highest-scoring fraction of target images (ρ_s = 0.5) is used for self-training.
4. Region-aware regularization for expert specialization. A mutual-information loss between region labels and expert selections, combined with an auxiliary region classifier loss, prevents the experts from collapsing into redundant behavior.

Main Findings

• GRAM outperforms all evaluated TTA baselines in every test city. On mIoU, GRAM reaches 0.859 in Dar es Salaam (vs. 0.844 for the next-best baseline, BeCoTTA, at 0.741), 0.870 in Kampala (vs. 0.844), and 0.907 in Maputo (vs. 0.904). F1-scores follow the same pattern: 0.921, 0.927, and 0.951 respectively.
• MoE source training alone already improves generalization. MoE Source beats Vanilla Source in every city (mIoU 0.806 vs. 0.681 in Dar es Salaam; 0.800 vs. 0.716 in Kampala; 0.900 vs. 0.800 in Maputo), showing the architecture — not just the adaptation step — contributes.
• Gains are largest on the slum class itself. In class-wise results (Table S1), Dar es Salaam slum IoU rises from 0.659 (MoE Source) to 0.752 (GRAM), and Kampala slum IoU from 0.720 (BeCoTTA) to 0.762 (GRAM), while non-slum performance stays high.
• The mutual-information loss is the most critical source-training component. Removing it drops Dar es Salaam mIoU to 0.734 and F1 to 0.823, a larger decline than removing the domain loss (0.836 mIoU) or removing filtering entirely (0.818 mIoU).
• Confidence-based pseudo-label filtering fails badly. Selecting pseudo-labels by confidence yields mIoU 0.463 and F1 0.501 on Dar es Salaam — far below both no filtering (0.818) and temporal-consistency filtering (0.837) — which the authors attribute to model overconfidence under domain shift.
• The region classifier picks visually and geographically sensible pairs. Jaccard similarity computed on CLIP features with K-means clustering shows classifier-selected pairs (e.g., Dar es Salaam–Cape Town, Kampala–Nairobi, Maputo–Cape Town) score higher than alternatives. Selected pairs yield higher mean IoU and slum IoU than the average of non-selected pairs in all three target cities (for example, Kampala–Nairobi: 0.868 mean IoU and 0.759 slum IoU, versus 0.784 and 0.605 for alternatives).
• GRAM also beats prior slum- and urban-village-specific baselines. Against TempSlum, UV-SAM, and LtCUV (Table S2), GRAM improves by approximately 4.0% in mIoU and 2.5% in F1-score on average across the three test cities.
• Temporal analysis reveals divergent trends across the three target cities. Applying the model to multi-temporal imagery, Kampala's slum share rose slightly from 8.4% in 2015 to 8.6% in 2023; Maputo's rose sharply from 35.3% in 2016 to 41.2% in 2023; Dar es Salaam's fell from 17.3% in 2015 to 12.6% in 2022.

Methodology in Plain English

The data. The authors gathered satellite imagery from ESRI World Imagery Wayback, cut into 256×256 tiles at zoom level 16 (approximately 1.2 meters per pixel, subject to latitudinal variation), covering 12 source cities: Cairo, Cape Town, Nairobi, and Ouagadougou in Africa; Colombo, Karachi, and Mumbai in Asia; Caracas, Medellín, and Rio de Janeiro in South America; and Port-au-Prince and Tegucigalpa in Central America. Ground-truth binary masks (slum = 1, non-slum = 0) came from the Atlas of Informality and city-specific datasets, manually generated by geography experts. To scale up supervision, they first trained a semi-supervised model based on the ST++ architecture on 2,714,489 image tiles (of which 86,752 have ground-truth annotations), then used its predictions as pseudo-labels to train a fully supervised baseline on a broader image set. Three held-out cities — Dar es Salaam, Kampala, and Maputo — totaling 529,633 tiles with 17,536 manual annotations, were reserved exclusively for adaptation and evaluation.

Phase 1 — source training. GRAM inserts lightweight MoE blocks into L intermediate layers of a transformer encoder, before the segmentation head. Each block holds a set of small MLP expert adapters. For each image, a gating network tied to that image's source region adds Gaussian noise to its routing logits, picks the top-k experts, and applies a softmax over just those top-k scores to weight their outputs. This lets different cities route their tokens to different experts, while a shared backbone learns what slums have in common everywhere. Two regularizers keep experts from becoming clones: a mutual-information term that rewards a strong statistical dependence between region and expert selection, and a small region classifier trained with cross-entropy. The total objective is the pixel-wise segmentation loss plus weighted versions of both regularizers.

Phase 2 — test-time adaptation. For each unlabeled target image, an external region classifier predicts which source region it most resembles, and the model generates a pseudo-label by routing through that region's experts. To judge whether the pseudo-label can be trusted, GRAM computes a stability score: the summed mean IoU between that pseudo-label and the pseudo-masks produced by routing through every other source region. The half of target images with the highest stability scores are kept (ρ_s = 0.5), and the model is fine-tuned on them with pixel-wise cross-entropy. No target labels are ever used — the paper emphasizes this is a fully unsupervised setting.

Settings. The backbone is SegFormer, trained with SGD at a learning rate of 0.0001 and momentum 0.99; the number of experts E is 12 and k is 2.

One inconsistency to flag: the Data section describes tiles at approximately 1.2 meters per pixel, while the dataset-release paragraph later states a spatial resolution of 10 meters per pixel at 256×256 pixels. The paper does not reconcile these two figures.

Why This Matters

Impact on research. The paper targets a well-known failure mode in geospatial machine learning: models trained on one region rarely transfer to another. Its contribution is showing that test-time adaptation, combined with expert specialization, can close much of that gap without any target labels — and providing a large, released multi-continent dataset (including both manual annotations and GRAM-derived masks across the 12 training and 3 testing cities) for others to build on. The authors report that, as of 2025, these 15 cities have an aggregated population of 120,174,837, with 215,148 informal settlement polygons identified by GRAM.

Real-world applications.

• Slum monitoring where census and survey data are missing. The framework produces pixel-level slum maps for cities where official statistics are scarce, costly, politically sensitive, or logistically difficult to collect.
• Tracking informal settlement change over time. Applied to multi-temporal imagery, it quantified slum-share shifts in Kampala, Maputo, and Dar es Salaam, giving planners year-over-year evidence rather than a single snapshot.
• Targeted resource allocation. "The ability to accurately identify areas of greatest need allows for a more strategic and targeted allocation of resources," the authors argue, which matters most in low- and middle-income settings.
• Policy evaluation. By publicly releasing a computation method, the authors aim to let local decision-makers assess the long-term impact of urban policies even without traditional census data.

Industry relevance. Satellite and geospatial analytics companies, humanitarian and development organizations, and municipal planning agencies all need scalable, label-efficient mapping pipelines. The result that confidence-based pseudo-label filtering collapses (mIoU 0.463) while consistency-based filtering holds up (0.859 in the full model) is a directly transferable lesson for anyone building self-training systems under domain shift.

Future Directions

• Extending beyond three target cities and beyond Africa. The conclusion explicitly frames broader geographic coverage as future work; the current evaluation holds out only Dar es Salaam, Kampala, and Maputo.
• Reducing dependence on the external region classifier. Routing depends on a classifier trained without explicit geographic information; testing how robust adaptation is when that classifier is wrong, or replacing it with an unsupervised matching step, is an open question.
• Reconciling and documenting the released data specification. The stated tile resolution differs between sections, and the appendix dataset details are truncated in the available text, so the exact composition of the released imagery remains to be confirmed from the repository.
• Turning maps into policy evidence. The paper shows divergent slum trends in three countries with comparable economic growth; determining which policies explain those divergences — and validating the tracked trends against ground observation — is the natural next step.

Target Audience

Researchers and practitioners in remote sensing, computer vision, and computational social science working on domain adaptation, test-time adaptation, or MoE architectures, especially those applying deep learning to satellite imagery for urban and development economics. It is also relevant to urban planners, geospatial analysts, and humanitarian organizations seeking label-efficient methods for mapping informal settlements, and to policymakers interested in data-driven urban metrics where official statistics are unavailable. Readers without a background in segmentation or domain adaptation will find the methods section demanding; the introduction, discussion, and temporal-analysis results are accessible to a broader audience.

Paper details: arXiv:2511.10300v1 [cs.CV], 13 Nov 2025, licensed CC BY-NC-SA 4.0. Affiliations: Max Planck Institute for Security and Privacy (MPI-SP) and KAIST. Code and datasets: https://github.com/DS4H-GIS/GRAM. Funding acknowledged from the National Research Foundation of Korea (NRF) via MSIT (No. RS-2022-NR068758 and RS-2025-00563196).

Authors’ abstract

Satellite-based slum segmentation holds significant promise in generating global estimates of urban poverty. However, the morphological heterogeneity of informal settlements presents a major challenge, hindering the ability of models trained on specific regions to generalize effectively to unseen locations. To address this, we introduce a large-scale high-resolution dataset and propose GRAM (Generalized Region-Aware Mixture-of-Experts), a two-phase test-time adaptation framework that enables robust slum segmentation without requiring labeled data from target regions. We compile a million-scale satellite imagery dataset from 12 cities across four continents for source training. Using this dataset, the model employs a Mixture-of-Experts architecture to capture region-specific slum characteristics while learning universal features through a shared backbone. During adaptation, prediction consistency across experts filters out unreliable pseudo-labels, allowing the model to generalize effectively to previously unseen regions. GRAM outperforms state-of-the-art baselines in low-resource settings such as African cities, offering a scalable and label-efficient solution for global slum mapping and data-driven urban planning.