LoCA: Enabling Multi-Modality in Geospatial Location Encoders

AI summary

Overview

Research area: Geospatial machine learning and representation learning — specifically location encoders, models that map a latitude/longitude pair to a general-purpose embedding vector, trained by contrastive alignment against Earth Observation (EO) imagery.

Technical level: Advanced. The paper combines a measure-theoretic argument about intrinsic dimension (Hausdorff dimension, Lipschitz maps, pushforward measures) with a large empirical benchmark, though its central story can be followed with intermediate background.

Scope: A theoretical diagnosis of why contrastive location encoders are structurally mismatched to their training objective, plus a two-stage pipeline (LoCA) that extends them to Sentinel-1 radar, Sentinel-2 optical imagery and tabular metadata, evaluated on 17 downstream tasks against five state-of-the-art baselines.

What This Paper Is About

Location encoders like SatCLIP and GeoCLIP learn a function from coordinates to a high-dimensional vector by pulling each coordinate's embedding toward the embedding of a satellite image taken at that location. The authors prove this setup can never reach the optimum of the contrastive objective, because a smooth map from a 2-dimensional coordinate sphere can only produce a 2-dimensional output surface inside the 255-dimensional embedding sphere. The goal is to explain why existing encoders nonetheless work, identify what a training target must satisfy to support multiple sensors, and build a pipeline that meets those conditions.

Key Contributions

1. A proof that location encoders constrain contrastive learning away from its optimum. Theorem 1 shows that any Lipschitz location encoder maps the coordinate sphere onto a set of intrinsic (Hausdorff) dimension at most k (with k = 2 for geographic coordinates), that its output distribution is mutually singular with the uniform target distribution on the embedding sphere, and that the contrastive loss of any aligned pair strictly exceeds the unconstrained minimum. Corollary 1 extends this to the entire bandwidth-L SatCLIP family.

2. An empirical demonstration of spectral collapse, and a reinterpretation of existing encoders. Joint training from scratch collapses the embedding covariance to extremely low rank; the authors recast SatCLIP and GeoCLIP as adapters over a frozen, pretrained image representation, where the frozen encoder acts as a covariance regularizer supplying a high-rank target.

3. Two requirements for multimodal location encoding. (i) Co-alignment: independently pretrained encoders do not place paired inputs at the same point, and MLP adapters cannot manufacture alignment that is absent. (ii) High effective rank: the location encoder inherits its target's spectrum, so masked-image-modeling-based geospatial foundation models are unsuitable anchors because their variance concentrates on few directions.

4. The LoCA pipeline. Stage 1 jointly trains Sentinel-1 and Sentinel-2 encoders with DHEL to produce a co-aligned, high-rank multimodal target; Stage 2 freezes both image encoders and fits a location encoder and a 14-variable tabular metadata encoder as adapters using the multi-view MV-DHEL objective. Against five state-of-the-art encoders on 17 tasks, LoCA has the most task wins and wins 68.8% of all per-task comparisons.

Main Findings

• Hard dimension ceiling: A 255-dimensional embedding sphere is used along only a 2-dimensional surface, regardless of architecture, bandwidth or parameterization. The location embedding lies on a 2-dimensional surface inside a 255-dimensional sphere.

• Joint training collapses the spectrum: A SatCLIP-style model trained from scratch without a pretrained anchor reaches an effective rank of only 8 (3.1% of the representation dimensionality). With a MoCo ResNet-50 anchor, SatCLIP reaches 89 (34.9%); Copernicus-FM, anchored on DINOv2, reaches 158 (20.6%); LoCA's Stage 1 reaches 158 but 62% of its available dimensions.

• Freezing is what makes these models work: Because the geometric image stays 2-dimensional while the empirical covariance tracks the frozen target's spectrum, geometry and spectrum decouple. Existing encoders behave as coordinate-indexed lookup tables rather than as jointly learned representations.

• Co-alignment is measurable and necessary: The two Stage 1 encoders show cosine similarity 0.94 and alignment loss 0.12, whereas pairing two independently pretrained encoders for the same modalities gives alignment loss 1.00 (cosine 0.50). Removing Stage 1 and substituting two independently pretrained MoCo encoders collapses mean linear-probe performance by 56.9 (Biodiversity & Ecology), 53.6 (Land cover), 26.8 (Wildfire), 61.3 (Geophysical) and 21.1 (Environmental) points relative to the full model.

• LoCA leads the benchmark: Best on 8 of 17 tasks, top-2 on 11 of 17, best under linear probing on 8 of 12 tasks. Head-to-head wins: 5/9 vs AlphaEarth, 11/17 vs Copernicus-FM, 12/17 vs SatCLIP, 12/17 vs GeoCLIP, 13/17 vs GT-Loc.

• Where LoCA wins and loses: It leads on reBEN land cover, Countries, iNaturalist, Biome, Ecoregions, and on Median Income among socio-economic tasks. AlphaEarth leads SatBird and Wildfire forecasting, where LoCA is second by 0.3%. SatCLIP leads Population, MSLP and Relative Humidity; AlphaEarth leads Air Temperature and Elevation; GT-Loc and GeoCLIP lead California housing.

• Clipboard caveat on competitors: GT-Loc and GeoCLIP's high effective rank is described as misleading for Earth Observation, since their CLIP-derived encoders inflate it with irrelevant language information; AlphaEarth's effective rank is unavailable because its embeddings were only partially retrievable.

• Metadata encoder adds modest value: Removing it changes mean linear-probe score by −0.1 (Biodiversity & Ecology), +0.5 (Land cover), +0.3 (Wildfire), −3.5 (Geophysical) and −1.3 (Environmental).

• Stage 2 transfers to supervised encoders: On 10 single-label land-cover classes, adding LoCA's Stage 2 raises mean F1 from 74.7 to 76.6 for a location-only encoder, 69.6 to 75.9 (Sentinel-1), 78.4 to 80.9 (Sentinel-2), 78.1 to 80.8 (S1+S2) and 80.9 to 84.3 (all concatenated embeddings).

Methodology in Plain English

The authors first treat the coordinate-to-embedding function as a mathematical object and ask what its output can possibly look like. Because a smooth function cannot inflate dimension, two numbers of input (latitude, longitude) can only paint a 2-dimensional sheet inside a 255-dimensional sphere — and the ideal target for contrastive learning is spread uniformly over the whole sphere. They prove these two sets do not overlap, so the loss must sit strictly above its ideal value; they then check what happens in practice by measuring the effective rank of trained embeddings.

That diagnosis suggests a fix: keep one side of the pairing frozen, so the trainable side has a rich, fixed cloud of targets to fold onto rather than collapsing together with a partner. The authors extend this to multiple sensors in two stages. Stage 1 trains the Sentinel-1 (radar) and Sentinel-2 (optical) image encoders together with DHEL, so that matched images land at the same point in a shared space — co-alignment becomes a property of the loss solution rather than something bolted on afterward — and so the space is used broadly rather than collapsing. Stage 2 freezes both image encoders, attaches a small projection to each, and trains a location encoder (spherical harmonics of degree 40 followed by a 2-hidden-layer SIREN) plus a metadata encoder (a 3-layer MLP over 14 variables such as elevation, land cover and population density) with the multi-view MV-DHEL loss. At inference, only a coordinate query is needed; the satellite encoders are not required.

They build on the S2-100K SatCLIP training set, adding co-located Sentinel-1 imagery and the 14 metadata variables, use ResNet-50 backbones for all LoCA image encoders, share a 2-layer projection head across modalities, and train both stages for 100 epochs each. Evaluation is on frozen embeddings: a LightGBM LambdaRank objective for the five climate-ranking tasks, and identical linear and MLP probes for everything else, averaged over 10 seeds (3 for SatBird).

Why This Matters

Research impact. The paper supplies a theoretical reason why coordinate-based encoders behave as they do, shifting the framing from "contrastive representation learning" to "fitting an inverse map onto a frozen point cloud." It turns that framing into concrete engineering requirements (co-alignment, high effective rank) that future multimodal encoders can be checked against, and it cautions against using masked-image-modeling foundation models as anchors when their spectra are collapsed.

Real-world applications:

• Wildfire danger forecasting (via Mesogeos) — location embeddings as inputs to danger models in Mediterranean landscapes.
• Biodiversity and conservation — species distribution modeling and bird encounter-rate prediction (SatBird), plus iNaturalist, biome and ecoregion classification.
• Land cover and environmental monitoring — reBEN land-cover classification and ordering of long-term climate variables from SeasFire.
• Socio-economic analysis — median income and population estimation, and housing-price prediction.

Industry relevance. Because a coordinate query alone returns an embedding, deployments do not need to store or re-fetch imagery, and embeddings remain queryable at any coordinate and any time — unlike published embedding products, which AlphaEarth releases as 64-dimensional annual embeddings at 10 m, Copernicus-FM as 0.25° multi-sensor averages, and TESSERA as annual int8 embeddings tied to a release window. That distinction matters for insurance, agriculture, conservation and disaster-response pipelines that need an on-demand vector for an arbitrary location.

Future Directions

• Closing the remaining gap to observation-based products. LoCA leads overall but trails AlphaEarth on SatBird and Wildfire forecasting (by 0.3% on the latter); combining location embeddings with imagery or with additional sensors is the obvious next step.
• Using more of the sphere. LoCA populates 158 of 255 dimensions (62%); pushing effective rank higher, or finding objectives that do so more reliably, remains open.
• Scaling beyond two sensors. The paper argues that the number of pairwise loss terms grows quadratically with modalities, and that adapters cannot align M independently trained encoders — a single multi-view objective over more sensors is the proposed route.
• Better handling of the metadata modality. Theorem 1 applies to the metadata encoder with k = 14, restricting its use as a Stage 1 anchor; how to exploit tabular variables during pretraining rather than only in Stage 2 is unresolved.

Target Audience

Researchers and practitioners working on geospatial representation learning, Earth Observation foundation models and contrastive learning, particularly those who build or consume coordinate-indexed embedding products. It is also valuable for applied scientists in conservation, climate and socio-economics who use location embeddings as frozen features, and for methodologists interested in how intrinsic dimension and effective rank constrain self-supervised objectives. Readers wanting only the practical takeaway can focus on the pipeline description and the benchmark table.

Authors’ abstract

Location encoders map geographic coordinates to high-dimensional descriptors through contrastive alignment with paired Earth Observation imagery. We prove that this construction is structurally limited: any Lipschitz location encoder produces an output of intrinsic dimension at most 2, regardless of architecture or bandwidth, which forces the contrastive loss to strictly exceed its asymptotic optimum. Empirically, joint training also collapses the encoder's embedding covariance to low rank. Although dimension-limited, current location encoders sidestep this spectral collapse by training against a frozen image encoder, which we reinterpret as a covariance regularizer that supplies a high-rank target for the location encoder to fold onto. However, extending this approach to multimodal inputs is challenging, as location encoders should simultaneously map a single location to multiple, potentially incompatible, descriptors. We identify two requirements a regularizer must satisfy for location encoding in multimodal regimes: co-alignment across modalities, since adapters alone cannot align independently pretrained encoders; and high effective rank, since the location encoder inherits its target's spectrum. We propose LoCA (Location Contrastive Alignment), a two-stage pipeline that satisfies both. Stage 1 jointly trains Sentinel-1 and Sentinel-2 encoders with contrastive learning, producing a co-aligned, high-rank multimodal target. Stage 2 freezes the image encoders and trains a location and a tabular metadata encoder as projection adapters with multi-view contrastive learning. Against five state-of-the-art encoders on 17 tasks ranging from land cover and wildfire forecasting to geophysical, socio-economic, biodiversity and climate applications, LoCA achieves the most task wins, is the only model to outperform every other on a majority of tasks, and wins 68.8% of all per-task comparisons.