CrossTimeEdit: A Decade-Spanning Cross-View Dataset and Reward-Guided Editing for Historical Street-View Generation

AI summary

Overview

Research area: Computer vision — cross-view (satellite–street) image synthesis, instruction-based image editing, and reinforcement-learning post-training of diffusion/flow-matching generative models, with an application to urban spatio-temporal analysis.

Technical level: Advanced. The paper assumes familiarity with diffusion/flow-matching generative models, LoRA fine-tuning, GRPO-style reinforcement learning, and vision-language-model (VLM) scoring.

Scope: The paper introduces VIGOR-his, a 43,653-quadruplet decade-spanning cross-view dataset across 11 cities, and CrossTimeEdit, a street-view editing model trained on it with supervised fine-tuning plus multi-reward online RL to reconstruct plausible historical street views from recent ones.

What This Paper Is About

Historical street-level imagery is sparse and unevenly collected, so for many locations there is no photograph of what a street looked like a decade ago. The authors argue that generating such a view should not be done from scratch out of random noise (as satellite-to-street models do), because satellite imagery constrains facades, textures, and occluded structures too weakly, and models end up altering buildings and roads that never changed. Their goal is to recast the problem as local editing: take the recent street view as an anchor for viewpoint and unchanged appearance, and use the differences between earlier and recent satellite images as evidence of what to modify.

Key Contributions

1. VIGOR-his dataset. 43,653 location-level cross-view quadruplets spanning roughly a decade, across 11 cities on three continents, built with an automated pipeline covering spatial pairing, viewpoint screening, change classification, cross-view and temporal consistency screening, satellite-based change description, and local editing instruction generation with validation. It adds change categories, satellite-grounded change descriptions, and satellite-grounded local editing instructions that prior cross-view datasets do not jointly provide.

2. A three-criterion VLM reward and evaluation protocol. Instruction Alignment (IA), Background Preservation (BP), and Quality and Physical Plausibility (QP), each scored 0–10 by Gemini-3.1-Flash-Lite, combining earlier-target edit verification, recent-input preservation assessment, and panorama-aware quality checks. The same protocol serves as both the RL reward and the offline editing benchmark.

3. CrossTimeEdit training recipe. Starting from FLUX.2 [Klein] 4B, the authors apply attention-projection LoRA supervised fine-tuning followed by online RL with Flow-GRPO, augmented with Group reward-Decoupled Normalization Policy Optimization (GDPO), which they call dimension-wise group normalization (DGN) — normalizing each reward dimension before weighted aggregation (weights 0.45 IA, 0.30 BP, 0.25 QP).

4. Empirical gains. A 17.12% improvement in Editing Overall over the pretrained FLUX.2 [Klein] 4B baseline (6.800 to 7.964) and a 33.39% improvement in IA, with the highest IA, QP, and Overall among the evaluated open-source editing models and substantially higher cross-view scores than Sat2Density, ControlS2S, and ControlNet.

Main Findings

• Editing gains over the backbone: CrossTimeEdit reaches IA 7.950, BP 8.264, QP 7.631, and Overall 7.964, versus 5.960 / 8.008 / 6.861 / 6.800 for FLUX.2 [Klein] 4B — a 17.12% Overall improvement and a 33.39% IA improvement.

• Closed-source models still lead on preservation: GPT-Image-2 scores Overall 8.581 with BP 9.633, Nano Banana 2 Lite 8.302 (BP 9.161), and Qwen-Image-3.0-Pro 8.042. The authors state explicitly that background preservation (BP) remains a key limitation of their model, even though its IA and QP approach closed-source levels.

• Cross-view comparison is lopsided on VLM metrics but not on pixel metrics: CrossTimeEdit scores CV-C 3.005, CV-VR 3.980, CV-PQ 3.998, Overall 3.546 against Sat2Density (2.015 Overall), ControlS2S (2.084), and ControlNet (1.740). Yet its PSNR (12.589), SSIM (0.463), and SD (18.705) are lower than Sat2Density (14.685 / 0.532 / 20.040) and ControlS2S (14.119 / 0.519 / 20.015).

• Human study supports the VLM protocol: A human preference study reported in Appendix B.4 finds that per-model pairwise preference rates correlate positively with both Editing Overall and Cross-view Overall. The authors conclude that pixel-, structure-, and edge-based similarity metrics alone are insufficient for judging historical street-view generation.

• RL adds what SFT alone breaks: SFT alone (w/o RL) raises IA to 6.919 but drops BP to 6.698 and QP to 6.543 (Overall 6.759, below the 6.800 baseline). RL on top restores and exceeds preservation and quality while keeping the IA gain.

• RL without SFT underperforms: Applying RL directly to the pretrained backbone (w/o SFT) gives 6.085 / 8.112 / 6.886 / 6.893, showing that SFT initialization matters.

• Ground truth in the IA reward matters: Removing the earlier street view from the IA reward (w/o GT) yields 7.938 / 7.356 / 7.508 / 7.656 — slightly higher IA but lower BP, QP, and Overall, consistent with weaker target grounding encouraging overly broad plausible edits.

• Dimension-wise normalization matters: Removing DGN gives a slightly higher IA (8.148) but lower BP (7.433), QP (7.432), and Overall (7.755), indicating poorer balance across objectives.

• Dataset scale relative to prior work: VIGOR-his provides 43,653 quadruplets over 3 continents and is the only dataset in the comparison table marked as providing cross-view pairing, temporal street views, temporal satellite views, cross-view time alignment, and text modality; CityPulse has 757 time series but no satellite pairing, and VIGOR has 105,214 panoramas but no temporal pairs.

Methodology in Plain English

Building the data. The authors start from VIGOR, a cross-view geo-localization dataset covering Chicago, New York, San Francisco, and Seattle, and use its panorama identifiers to retrieve panoramas from roughly a decade earlier (2008–2009) against recent ones from 2019. They add seven more cities — Barcelona, Budapest, Copenhagen, Detroit, Los Angeles, London, and Sydney — sampling observations at 100 m intervals within OpenStreetMap study regions, with earlier imagery from 2012–2015 and recent imagery from 2022–2025. Earlier and recent panoramas are matched within 5 m of Haversine surface distance (Earth radius fixed at 6,371,000 m); the added cities are first grouped into 25 m clusters with a 25 m midpoint-suppression rule to avoid near-duplicate pairs. Panoramas are 2048×1024 and satellite crops 512×512, with north centered on each panorama via heading metadata.

Filtering and annotating with VLMs. Gemini-3-Flash-Preview rejects pairs with severe heading misalignment and labels each sample as building change, road change, building-and-road change, or no change. GPT-5.6-Luna scores street–satellite consistency at both periods, and only quadruplets scoring at least 7 on a 0–10 scale are kept. Because the earlier street view is unavailable at generation time, change evidence is taken from the satellite pair: Gemini-3.1-Pro-Preview describes structural changes along the four cardinal directions, then combines those descriptions with the recent panorama to write per-direction instructions using a MODIFY / PRESERVE vocabulary. Gemini-3.1-Flash-Lite validates each instruction on Grounding, Consistency, and Preservation, again requiring at least 7 on all three.

Reformulating generation as editing. The model receives the recent street view plus the instruction and must output the earlier street view; the true earlier view is used only for supervision and scoring. For no-change samples, all four directions are labelled PRESERVE and the target is the input itself, teaching the model to leave things alone.

Training. Stage one is supervised fine-tuning of attention-projection LoRA adapters (rank 32, α = 32) with the backbone, MLPs, and encoder/decoder frozen, over two five-epoch phases at learning rates 3×10⁻⁵ (412-step warmup, constant) and 5×10⁻⁵ (300-step warmup, warmup–stable–decay). Stage two is online RL: Flow-GRPO samples multiple candidate trajectories per condition, a VLM scores each finished image on IA, BP, and QP (no gradients flow through the evaluator), and the policy is updated with a PPO-style clipped objective. Instead of averaging the three raw scores, each dimension is normalized within its group first, then combined with fixed weights, then batch-normalized and clipped — the DGN step that stops one high-variance reward from dominating. RL runs 240 epochs at 7.5×10⁻⁶ on 2,000 randomly chosen changed training conditions.

Splits. The training set holds 21,993 samples (8,797 changed, 13,196 no-change, roughly 4:6); validation has 1,095 changed samples and the test set 1,094.

Why This Matters

Impact on research. The paper reframes historical street-view synthesis from "generate from satellite" to "edit the recent view using satellite change evidence," and shows that a multi-dimensional VLM reward combined with dimension-wise normalization produces a better balanced policy than a single holistic score. It also provides a dataset that is, per its own comparison table, the first to combine temporal street views, temporal satellite views, time alignment, and text in one resource — which turns an under-supervised problem into a supervised one.

Real-world applications:

• Urban planning and redevelopment monitoring — reconstructing how a streetscape looked before demolition or road reconfiguration, complementing overhead-only change assessment.
• Real-estate and insurance due diligence — plausible visual context for a property's surroundings at a past date, where no street-level capture exists.
• Cultural heritage and archival presentation — illustrating disappeared buildings at street level for public-facing historical narratives, with the paper's explicit caveat that outputs are hypotheses, not archival records.
• Scene-level modeling support — supplying street-level context in places where satellite imagery is plentiful but street imagery is not.

Industry relevance. The pipeline is a template for building instruction-tuning data at scale using VLM annotators plus strict consistency thresholds, and the GDPO/DGN integration into Flow-GRPO is a reusable recipe for anyone balancing multiple, differently-scaled aesthetic or task rewards. Mapping, autonomous-driving simulation, and digital-twin vendors are the natural adopters.

Future Directions

• Preservation is the weak point. BP and Overall trail all three closed-source models in Table 2, so closing that gap — rather than pushing IA further — is the stated priority.
• Historical details invisible from orbit. Changes that leave no satellite footprint (facade materials, signage, street furniture) remain fundamentally unobservable in the current formulation, and the authors list this as an open challenge.
• Viewpoint drift between capture periods. Residual differences in camera location and height between the two acquisition dates can be mistaken for temporal change; richer handling of this is left open.
• Evaluator dependence. Because training rewards and evaluation share the same Gemini-3.1-Flash-Lite protocol, the authors propose independent reward and evaluation models to avoid evaluator-specific preferences.
• Beyond unidirectional recovery. The conclusion sketches bidirectional and multi-temporal street-view generation as extensions of the framework.

Target Audience

Researchers and engineers working on diffusion/flow-matching image editing, cross-view synthesis, or RL post-training of generative models; practitioners building geospatial or urban-analytics datasets who want a worked example of VLM-driven annotation with explicit quality gates; and applied teams in mapping, digital twins, or insurance who need to reason about street scenes at dates with no direct imagery. Readers without background in flow matching or policy-gradient methods will find Sections 3 (dataset) and 5 (results) accessible, while Section 4 requires prior familiarity with GRPO-style optimization.

Authors’ abstract

Historical street-view imagery records urban evolution, but uneven coverage leaves substantial gaps in historical records. Generating plausible past appearances requires restoring changed structures while preserving persistent scene content. We construct VIGOR-his, a decade-spanning cross-view dataset containing 43,653 location-level quadruplets across 11 cities on three continents. Its automated pipeline performs spatial pairing, consistency screening, change classification, and the generation and validation of satellite-based change descriptions and local editing instructions. Based on VIGOR-his, we propose CrossTimeEdit, a model that reformulates historical street-view generation as editing, using recent street views to constrain viewpoint and unchanged appearance and temporal satellite differences as change evidence. Starting from FLUX.2 [Klein] 4B, we train CrossTimeEdit through supervised fine-tuning (SFT) followed by online reinforcement learning (RL). We design three street-view editing criteria, namely Instruction Alignment (IA), Background Preservation (BP), and Quality and Physical Plausibility (QP), as both RL reward dimensions and evaluation metrics. We optimize this multi-reward objective using Within Group Relative Policy Optimization for flow-matching models (Flow-GRPO) with Group reward-Decoupled Normalization Policy Optimization (GDPO), which normalizes each reward dimension before aggregation. CrossTimeEdit improves overall performance across the three editing criteria by 17.12\% over the pretrained baseline and outperforms cross-view generation models in scene consistency, visual realism, and perceptual quality. The implementation code, dataset, and model weights are available at https://luhanwen67.github.io/CrossTimeEdit-release/.