SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards

AI summary

Overview

• Research area: Multimodal large language models (MLLMs), spatial reasoning, scene graph generation (SGG), and reinforcement learning from verifiable rewards (RLVR).
• Technical level: Advanced. The paper assumes familiarity with reinforcement learning (GRPO, PPO-style clipping, KL regularization), reward shaping, bipartite matching with CIoU, and MLLM post-training.
• Scope: The paper introduces SpatialThinker, an MLLM that generates question-focused scene graphs inside its own reasoning chain and is trained with a dense, lexicographically gated multi-objective reward.

What This Paper Is About

Multimodal LLMs can describe images well but remain weak at spatial reasoning, such as judging which object is closer to the camera or how two objects relate in 3D. Existing spatial MLLMs tackle this with very large training sets, explicit 3D inputs (depth maps, point clouds), architecture changes, or reinforcement learning that rewards only the final answer. SpatialThinker instead teaches a model to build a mental scene graph of task-relevant objects, bounding boxes, and relations as an intermediate step, then reason over it to an answer, using a dense reward that scores formatting, object counts, answer accuracy, and bounding-box localization.

Key Contributions

1. SGG-grounded reasoning in a single pass: SpatialThinker is described as the first MLLM (to the authors' knowledge) to unify scene graph generation and visual reasoning via online RL, integrating scene graph construction inside the reasoning chain rather than treating SGG as a separate preprocessing step.
2. STVQA-7K dataset: A high-quality spatial VQA training set of 7,587 multiple-choice question-answer pairs synthesized from Visual Genome scene graphs, with a scalable pipeline supporting up to approximately 108K samples and dual-LLM verification for quality control.
3. Dense, lexicographically gated reward: A four-component reward (format, count, accuracy, spatial) ordered as format ≻ {count, accuracy} ≻ spatial, which enforces region-focused grounding during RL and also improves broad visual perception.
4. Released models at three scales: SpatialThinker at 3B, 7B, and 30B, with SpatialThinker-30B reported to surpass GPT-5 and Claude 4 Sonnet on average across 14 benchmarks.

Main Findings

• SpatialThinker-7B average accuracy of 70.5% across 14 benchmarks: This is +7.7 over its base model (Qwen2.5-VL-7B), +5.5 over an SFT baseline, and +3.2 over a sparse-reward GRPO baseline, while essentially matching GPT-5 (-1.0 average) and beating GPT-4o (+4.7), Claude 4 Sonnet (+1.8), and Claude 3.5 Sonnet.
• Dense rewards beat sparse rewards in every comparison: Vanilla GRPO improves the 7B base model by +4.4 and the 3B base model by +5.2, whereas the dense spatial reward raises this to +7.7 and +9.1 respectively (roughly 1.7×). The abstract states SpatialThinker-7B achieves 3.6× larger gains over SFT and 1.7× better in- and out-of-distribution generalization than sparse RL.
• SpatialThinker-30B reaches 74.5% average across 14 benchmarks: This surpasses GPT-5 by +3.0 and Claude 4 Sonnet by +5.8, and improves over its Qwen3-VL-30B base by +6.4.
• Strong 3D benchmark results: SpatialThinker-30B reaches 93.6% on the CV-Bench 3D split (+3.3 over GPT-5), while SpatialThinker-7B reaches 56.4% on 3DSRBench (surpassing GPT-4o by +12.1).
• Fewer data and no depth inputs: SpatialThinker-7B is trained on only 7K samples and RGB inputs, yet the paper reports it outperforming Spatial-RGPT-7B (which uses depth inputs and 700K training samples) by +10.3 on BLINK tasks, and SpatialBot by +11.5.
• Gains transfer to real-world and general VQA: SpatialThinker-7B scores 65.9% on MM-Star (+2.0 over base), 81.7% on VStarBench (+5.8 over base, +15.7 over GPT-4o), 76.3% on RoboSpatial-Home (+5.7 over base, +7.9 over GPT-4o), 66.4% on HallusionBench (+13.5 over base, +11.4 over GPT-4o), and 48.3% on MME-RealWorld-Lite (+4.2).
• Out-of-distribution generalization favors dense rewards: On the 13 held-out benchmarks, SpatialThinker-7B gains +8.6 on spatial VQA and +5.3 on real-world VQA over its base, versus +5.9 and +1.9 for sparse-reward GRPO and +0.9 and +2.8 for SFT. The paper describes this as nearly 3× the real-world gains of sparse-reward GRPO.
• Naive spatial rewards cause reward hacking: In the reward ablation on STVQA-7K val, adding a spatial reward on top of format and accuracy drops performance to 23.7% (from 74.9%), because models overgenerate boxes to exploit the CIoU reward. Adding the count reward recovers performance to 61.7%, and adding lexicographic gating and Region-of-Interest filtering brings it to 76.3%.
• Dual-LLM dataset filtering helps: Filtering the dataset with pass@2 correctness verification raises STVQA-7K val accuracy from 76.3% to 87.9%, a +13.0 improvement. The paper elsewhere states this cross-model verification yields +13% accuracy.
• Modest inference overhead: Because the model generates question-focused subgraphs rather than exhaustive scene descriptions, the scene graph component adds on average approximately 120 additional tokens.

Methodology in Plain English

The authors take three off-the-shelf multimodal models (Qwen2.5-VL-3B, Qwen2.5-VL-7B, and Qwen3-VL-30B) and train them with reinforcement learning only, without any supervised fine-tuning on STVQA-7K beforehand.

They first build a training set: starting from human-annotated Visual Genome scene graphs, they use Claude Sonnet 4 to write multiple-choice spatial questions and answers, then use GPT-4o to check them via pass@2 agreement. From an initial pool of 56,224 questions, the top 7,587 are kept based on rating, difficulty, and verification. They also extend the standard 50-predicate set with 34 additional spatial relations covering distance, size, orientation, and containment. For each question they extract a question-aligned subgraph of relevant objects and relations, keeping bounding boxes in absolute pixel coordinates.

During training, the model must produce a structured response with tags for observing the scene, writing a scene graph (as parseable JSON with object IDs, boxes, and subject-predicate-object triplets), thinking, and answering. The reward has four parts: a format reward (weight 0.1) that checks tag usage and JSON validity; an accuracy reward (weight 0.5) for exact match on the multiple-choice answer; a count reward (weight 0.2) that penalizes over- or under-generating objects and relations relative to ground truth (with object and relation sub-weights of 0.7 and 0.3); and a spatial reward (weight 0.2) computed only when the answer is correct, which matches predicted and ground-truth objects using the Hungarian algorithm and scores them by CIoU on bounding boxes. Matching costs combine IoU and label similarity with weights of 1.0 and 2.0. The rewards are ordered lexicographically: the model must format correctly first, then optimize count and accuracy, and only receives the spatial reward if the final answer is correct.

Optimization uses Group-Relative Policy Optimization (GRPO) with 8 rollouts per query, a sampling temperature of 1.0, a maximum context length of 16,384 tokens, rollout batch size 512, and global batch size 128. Training runs for 75 steps (5 episodes) on 4× NVIDIA H100 80GB GPUs, taking approximately 13 hours for the 3B model and 15 hours for the 7B model. The 30B model is trained with LoRA of rank 64 via the Tinker API. All parameters, including the vision encoder, are updated, using AdamW with bf16 precision, a learning rate of 1×10⁻⁶, weight decay of 1×10⁻², and a KL penalty coefficient of 10⁻². Models are trained and evaluated on images from 512×512 to 2048×2048 pixels. STVQA-7K uses a 90/10 train-validation split.

Evaluation covers 14 benchmarks in a zero-shot, greedy-decoding setting: eight spatial benchmarks (CV-Bench 2D and 3D, BLINK Spatial Relations and Relative Depth, 3DSRBench, MMVP, SpatialBench, SpatialReasonerEval, MindCube-tiny, and the held-out STVQA-7K validation split) and six real-world or general VQA benchmarks (VStarBench, RealWorldQA, MME-RealWorld-Lite, RoboSpatial-Home Configuration and Compatibility, MM-Star, and HallusionBench).

Why This Matters

The paper argues that structured spatial grounding with reward-aligned reasoning can produce robust spatial understanding from a small, high-quality dataset, rather than relying on hundreds of thousands to billions of samples, explicit 3D inputs, or architecture-specific modifications. It also reports that spatial rewards improve generic visual perception and hallucination robustness, not just spatial tasks, suggesting a broader benefit from grounding-based RL.

Real-world applications implied by the paper:

• Robotic manipulation and embodied AI, where the paper states precise spatial awareness underpins real-world deployment.
• Navigation, which the paper lists as a capability enabled by spatial reasoning.
• Augmented reality, also cited as an application area.
• Practical perception settings such as autonomous driving, remote sensing, and surveillance, which the paper identifies as domains covered by its real-world evaluation suite.

Industry relevance: the results position dense, structure-aware reward design as a data-efficient alternative to large-scale spatial data curation, and the reported 30B results claim superiority over GPT-5 and Claude 4 Sonnet on the 14-benchmark average, which matters for teams weighing proprietary APIs against open models that can be trained on 4 H100 GPUs in roughly 13 to 15 hours at the 3B and 7B scales.

Future Directions

• Scaling the synthesis pipeline: The authors state the pipeline can be extended to approximately 108K samples, the maximum supported by Visual Genome, enabling future large-scale post-training or RL fine-tuning.
• Scaling model size and training data jointly: The paper shows the same recipe transferring from 3B and 7B to 30B with LoRA; whether the trend continues at larger scales is left open.
• Testing generalization beyond the current benchmark suite: The paper reports emergent 3D reasoning from 2D rewards and 3D-linked relations, and mentions Appendix F demonstrating generalization to abstract and multi-view reasoning tasks, which invites further study of how far those priors transfer.
• Refining the reward design: The ablation shows how sensitive training is to reward components, including a severe reward-hacking failure when the spatial reward is added without counting and gating; the authors describe this as a staged reward-shaping process, implying further design work remains.

Target Audience

This paper is most useful for researchers and engineers working on multimodal LLM post-training, reinforcement learning with verifiable rewards, and spatial or embodied reasoning. It is also relevant to practitioners building robotics, navigation, or augmented-reality systems who need spatially grounded vision-language models, and to readers interested in scene graph generation as an in-loop reasoning representation rather than an offline annotation tool. A working knowledge of RL fine-tuning and multimodal model architectures is needed to follow the reward and optimization details.

Authors’ abstract

Multimodal large language models (MLLMs) have achieved remarkable progress in vision-language tasks, but continue to struggle with spatial reasoning. Existing spatial MLLMs rely on large-scale datasets, explicit 3D inputs, architecture-specific modifications, or sparse Reinforcement Learning (RL) methods that provide insufficient guidance for spatially-grounded reasoning. We introduce SpatialThinker. To our knowledge, it is the first MLLM unifying Scene Graph Generation (SGG) and visual reasoning in a single pass via online RL. The model simulates human-like spatial perception by constructing a mental scene graph of task-relevant objects and relations, and reasoning toward an answer via dense spatial rewards. Our contributions are threefold: (1) SGG-grounded reasoning: integrating SGG directly within the reasoning chain rather than as a disjoint preprocessing step; (2) STVQA-7K: a high-quality spatial VQA training dataset via a scalable synthesis pipeline; and (3) a dense spatial reward design that enforces structured grounding during RL and generalizes to improve broad visual perception. SpatialThinker-7B achieves 3.6$\times$ larger gains over SFT and $1.7\times$ better in- and out-of-distribution generalization than sparse RL. Trained on only 7K samples, SpatialThinker-7B matches GPT-5 and outperforms GPT-4o, while SpatialThinker-30B surpasses both GPT-5 and Claude 4 Sonnet on average across 14 spatial and real-world benchmarks, demonstrating that structured spatial grounding with reward-aligned reasoning enables robust spatial understanding with limited data.