GeoZero: Incentivizing Reasoning from Scratch on Geospatial Scenes

AI summary

Overview

Research area: Multimodal large language models (MLLMs) for remote sensing and geospatial scene understanding, with a focus on reinforcement-learning-based reasoning.

Technical level: Advanced. The paper assumes familiarity with supervised fine-tuning, group-relative policy optimization, reward shaping, and vision–language benchmark evaluation.

Scope: The paper introduces GeoZero, a two-stage (supervised fine-tuning then reinforcement learning) framework that trains a remote sensing MLLM to produce explicit reasoning traces without any pre-built chain-of-thought data, using two new datasets and a modified GRPO algorithm called A²GRPO.

What This Paper Is About

Existing remote sensing MLLMs that "think" before answering are usually cold-started on chain-of-thought (CoT) text that was generated by a general-purpose model such as GPT-4o and then verified or refined by hand. That process is expensive and injects human bias that may narrow the model's reasoning. GeoZero asks whether a model can develop genuine geospatial reasoning from scratch — learning to think purely from question–answer pairs and reinforcement learning feedback, with no CoT supervision at all.

Key Contributions

1. A CoT-free geospatial reasoning MLLM. The authors present GeoZero, which they describe as the first MLLM in the remote sensing community able to reason without any predefined CoT supervision, following a standard SFT–RL training paradigm built on Qwen3-VL-8B-Instruct.

2. Two new datasets. They aggregate a large collection, GeoZero-Raw (754,749 samples spanning scene classification, visual grounding, visual question answering, and image captioning), then derive GeoZero-Instruct (~610K samples) for SFT and GeoZero-Hard (~20K samples) for RL. Both contain only question–answer pairs, with no CoT data.

3. A²GRPO, a new RL algorithm. Answer-Anchored Group Relative Policy Optimization couples reasoning quality to answer correctness through an Answer-Modulated Thinking Reward (AMTR) and task-specific answer rewards, and it modifies the optimization objective with a binary mask that applies reference-model alignment only to answer tokens.

4. Demonstrated emergent reasoning across benchmarks. On UCM, AID, RSVG, DIOR-RSVG, RSVQA-HR, and RSICD, GeoZero surpasses or is competitive with prior remote sensing MLLMs while producing explicit reasoning traces it was never trained to imitate.

Main Findings

• Scene classification: GeoZero reaches 93.81% on UCM and 92.55% on AID, the best of the compared methods. For reference, the base Qwen3-VL-8B-Instruct scores 75.71% and 71.40%, VHM scores 91.70% on AID, RingMo-Agent scores 88.00% on UCM and 91.67% on AID, and TinyRS-R1 scores 90.20% on AID.

• Visual grounding: GeoZero attains 37.16% on RSVG, the best reported result there, and 75.67% on DIOR-RSVG. The base model scores 26.41% and 51.51%; the specialized grounding model GeoGround scores 26.65% and 77.73%; TinyRS-R1 scores 74.90% on DIOR-RSVG.

• Visual question answering: On RSVQA-HR, GeoZero reaches 74.46% on Presence and 83.59% on Compare, the highest on both question types (base model: 66.04% / 75.39%; VHM: 64.00% / 83.50%).

• Image captioning: On RSICD GeoZero achieves the best CIDEr (97.45) and METEOR (48.18), but its BLEU-4 of 29.17 is lower than SkyEyeGPT's 59.99. The authors note CIDEr is not explicitly optimized in the task-specific answer reward, suggesting the framework improves overall description quality beyond directly optimized components.

• SFT alone does not produce reasoning. In the ablation, SFT-only reaches 94.52% on UCM and 47.76% on RSVG but generates no thinking text; SFT+RFT reaches 95.24% and 47.68%, also without reasoning. Adding the RL stage (GeoZero) maintains 93.81% / 37.16% while activating reasoning.

• RL without SFT fails on hard tasks. RL-only training activates reasoning and scores 76.90% on UCM but collapses to 0.16% on RSVG, showing that SFT is needed to supply foundational geospatial knowledge.

• Task-specific RFT improves both. GeoZero+RFT reaches 95.48% on UCM and 50.04% on RSVG, with reasoning active in both cases.

• Hard samples matter. Replacing GeoZero-Hard with randomly selected data during RL drops GeoZero to 94.05% on UCM with no reasoning and 18.34% on RSVG with no reasoning; after RFT the random-data variant reaches 95.24% / 46.70%, still below the GeoZero-Hard variant's 95.48% / 50.04%.

• Both AMTR and the thinking mask are necessary. Removing both leaves GeoZero at 91.67% on UCM and 26.49% on RSVG with no reasoning. AMTR alone gives 93.10% / 28.28% (no reasoning); the thinking mask alone gives 93.57% / 26.57% (no reasoning); together they give 93.81% / 37.16% with reasoning active.

• General MLLMs barely reason on hard geospatial tasks. A diagnostic study (supplementary Section B) with Qwen3-VL-8B-Instruct shows that a reasoning system prompt raises UCM accuracy from 75.71% to 77.14% with a 53.10% thinking activation rate (80.72% correct on the reasoned subset, 223/420), but on RSVG accuracy drops from 26.41% to 24.04% and only 1 of 1,227 samples contained reasoning before the answer tag (0.08%). GeoZero reaches 99.52% activation on UCM (94.26% correct, 418/420) and 100% activation on RSVG (1227/1227).

• Reasoning length and quality correlate with accuracy. Analysis on the DIOR-RSVG test set shows accuracy increases with longer reasoning chains but drops slightly under overthinking, and accuracy correlates positively with the reasoning-quality score s_t.

• Evaluation protocol: Scene classification and VQA are measured by accuracy; visual grounding uses Acc@0.5, where a prediction counts as correct if the IoU with the ground truth exceeds 0.5; captioning is measured with BLEU-4, CIDEr, and METEOR.

Methodology in Plain English

The authors build GeoZero through three linked pieces.

Data aggregation and filtering. They pool publicly available remote sensing datasets into GeoZero-Raw, covering scene classification, visual grounding, VQA, and image captioning (754,749 samples total, with roughly 750K stated in the text). Each sample is rewritten into a uniform instruction-following format with a task descriptor such as [cls], [grounding], [vqa], or [caption], and optional task-specific hints are included with 50% probability, drawn from more than 20 variations per task. They then train a model on this pool to act as a data filtering model. Because samples mispredicted in a single pass are likely to be the hardest, they run every sample once and keep only the incorrect predictions, re-evaluate those candidates three times with different random seeds, rank them by average error rate, and keep the top portion with equal representation across the four tasks. That becomes GeoZero-Hard (~20K samples). All samples sharing an image with GeoZero-Hard are removed to prevent overlap, and the remaining samples form GeoZero-Instruct (~610K samples). A roughly 300K-sample validation subset, GeoZero-Raw-val, is reserved for the hard-sample procedure.

Two-stage training. Stage one is supervised fine-tuning on GeoZero-Instruct, which teaches basic geospatial knowledge but no reasoning style. Stage two is reinforcement learning on GeoZero-Hard, the pool of deliberately hard questions that simple pattern-matching cannot solve, which pushes the model toward deliberate reasoning.

The A²GRPO algorithm. The reward has two parts. The answer reward is task-specific and continuous, falling in [0, 1]. The thinking reward is gated by answer quality: it only contributes when the answer is good enough, via a sigmoid gate with threshold τ and sharpness k, and it is further multiplied by the answer reward. The combined reward is r = r_a + λ·r_t. On the optimization side, the authors change GRPO's objective so that alignment with the reference SFT model is enforced only on tokens inside the answer tags, using a binary mask; reasoning tokens are left free so the model can explore diverse thinking paths. They also simplify the prompt template: because <think> and < rarely appear at the start of sentences after SFT, the template keeps only the <answer></answer> markers, removing constraints that prevent the model from initiating reasoning.

Training setup. Qwen3-VL-8B-Instruct is the base model. SFT and RL each run for 1 epoch with global batch sizes of 64 and 48; learning rates are 1e-4 and 5e-6 with weight decays 0.1 and 0.01. The vision encoder stays frozen while the rest is fine-tuned with LoRA (rank 16, α = 32). Training uses bfloat16, DeepSpeed, and FlashAttention on 8 NVIDIA A100 GPUs. For A²GRPO, G = 8, temperature is 0.9, β = 0.04, the AMTR gate uses k = 12 and τ = 0.5, and λ = 0.3.

Why This Matters

The work argues that cold-start CoT annotation is both costly and cognitively limiting for remote sensing, and shows a path where reasoning emerges from the training signal itself. If reasoning can be induced without curated thinking traces, the annotation bottleneck for domain-specialized reasoning models weakens substantially, and the resulting reasoning traces are the model's own rather than an imitation of another model's — which the authors connect to transparent, explainable AI. The failure of a reasoning prompt alone on RSVG (only 1 of 1,227 samples reasoned) suggests general MLLMs cannot simply be prompted into geospatial reasoning, making trained approaches like this necessary.

Real-world applications implied by the tasks the model handles (the paper names the task types but does not enumerate deployments):

• Land-use and scene classification over satellite and aerial imagery, where the model classifies scenes before answering.
• Object localization and spatial grounding tasks such as finding specific structures or vehicles in overhead imagery, evaluated here via Acc@0.5 on RSVG and DIOR-RSVG.
• Natural-language querying of imagery through VQA, of the kind tested on RSVQA-HR's Presence and Compare question types.
• Automatic description generation for imagery archives, tested on RSICD captioning.

Industry relevance: Organizations that rely on remote sensing imagery at scale — earth observation providers, mapping platforms, environmental and agricultural monitoring, and defense or disaster-response analysts — depend on models that can justify their conclusions. A framework that produces inspectable reasoning before an answer, without paying for curated reasoning annotations, is directly relevant to those workflows.

Future Directions

• Understanding the reward's internals. The thinking-quality score s_t combines trajectory length, semantic diversity, and sentence redundancy, but the full formulation lives in supplementary Section F and the task-specific reward definitions in Section G, which are not included in the provided content. How these interact and how sensitive results are to the gate parameters k = 12 and τ = 0.5 remains an open question.

• Explaining the grounding gap. GeoZero leads on RSVG (37.16%) but trails GeoGround on DIOR-RSVG (75.67% versus 77.73%), and RL-only training collapses to 0.16% on RSVG. What limits reasoning-based spatial localization, and whether reasoning helps or hinders precise box prediction, is unresolved.

• Reconciling captioning behavior. GeoZero has the best CIDEr and METEOR but a BLEU-4 of 29.17 against SkyEyeGPT's 59.99, which suggests the reasoning process changes n-gram style more than overall description quality. Whether reward design can close that gap is not addressed.

• Scaling and generality. All experiments use one base model (Qwen3-VL-8B-Instruct) at 8B scale, and the paper does not report whether reasoning emerges at other model sizes or with other backbones. Extending the claim of "universal" emergent reasoning beyond these four task families is untested here.

Target Audience

Researchers and graduate students working on multimodal large language models, remote sensing vision–language models, and reinforcement learning for reasoning. It is also relevant to practitioners building geospatial AI systems who want reasoning-capable models without paying for chain-of-thought annotation, and to those interested in reward design for GRPO-style training beyond the language domain. Readers should be comfortable with policy-gradient methods and remote sensing benchmark conventions; the description of the two-stage pipeline and the dataset construction is accessible to a broader audience.

Authors’ abstract

Multimodal large language models (MLLMs) have undergone rapid development in advancing geospatial scene understanding. Recent studies have sought to enhance the reasoning capabilities of remote sensing MLLMs, typically through cold-start training with elaborately curated chain-of-thought (CoT) data. However, this approach not only incurs substantial annotation costs but also introduces human biases that may limit the diversity of model reasoning. To address these challenges, we propose GeoZero, a framework that enables MLLMs to perform geospatial reasoning without any predefined CoT supervision. Specifically, we construct two datasets, GeoZero-Instruct and GeoZero-Hard. GeoZero-Instruct allows the model to acquire preliminary geospatial knowledge through supervised fine-tuning, while GeoZero-Hard stimulates deep reasoning during the subsequent reinforcement learning stage. Furthermore, we introduce Answer-Anchored Group Relative Policy Optimization (A$^2$GRPO), where the reasoning process is regularized by the model's own answers, encouraging diverse yet accurate thinking. Extensive experiments on multiple remote sensing vision-language benchmarks demonstrate that GeoZero not only surpasses existing state-of-the-art methods but also fosters universal emergent reasoning capabilities across diverse geospatial tasks. Code, data, and models are available at https://github.com/MiliLab/GeoZero.