Synthetic Data for Robust Runway Detection

AI summary

Overview

Research area: Synthetic data generation and synthetic-to-real domain adaptation for object detection, applied to runway detection in vision-based aircraft landing.

Technical level: Intermediate. Familiarity with object detection (Faster R-CNN, COCO Average Precision) and basic domain adaptation concepts helps, but the methodology is explained clearly enough for a motivated non-specialist.

Scope: A single-case industrial study showing that simulator-generated images, mixed with a small set of annotated real images and combined with a customized CARE domain adaptation loss, improve runway detection accuracy and robustness to nighttime conditions on real validation images.

Published as arXiv:2510.20349v1 [cs.LG] on 23 Oct 2025, licensed CC BY 4.0. Accepted in Computer Analysis of Images and Patterns, September 2025 (CAIP 2025), with the Version of Record in Lecture Notes in Computer Science (https://doi.org/10.1007/978-3-032-04968-1_25).

What This Paper Is About

Deep vision models are ready for critical uses such as autonomous landing, but collecting and labeling enough real images for a single company or product is expensive, and rare conditions (night, snow, fog) are hard to capture safely. The authors ask whether synthetic images from a commercial flight simulator can fill that gap for runway detection, and whether a domain adaptation method can mitigate the mismatch between synthetic and real images. They also test robustness on a condition absent from the real training data: nighttime images of runways.

Key Contributions

1. Demonstrates the benefits of training a runway detection model with synthetic data in an industrial use case, using a private real dataset and a simulator-generated dataset.
2. Customizes the CARE domain adaptation method for runway detection by dropping its object-size and frequency reweighting terms, which are unnecessary in a single-class setting with consistent object size and aspect ratio, leaving the Faster R-CNN detection loss plus a weighted alignment loss.
3. Evaluates robustness under an adverse condition not represented in the real data, showing that models trained with synthetic nighttime images detect runways at night on real images while a real-data-only model largely fails.
4. Compares training strategies (real only, synthetic only, random mixing, the CARE sampler alone, and full CARE) on both daytime and nighttime real validation sets.

Main Findings

• Mixing beats real-only in daytime conditions. On the daytime validation set with models trained without synthetic nighttime images, AP was 58.60 for REAL, 59.04 for SYNTH (+0.44%), 65.25 for MIX (+6.65% compared to REAL only), 64.01 for SAMPLER, and 63.66 for CARE. The authors suggest the synthetic-only model still performs competitively because the backbone was pretrained on COCO2017 and already acquired features relevant to real images.
• Adding nighttime synthetic data changes daytime rankings. With synthetic nighttime images included, daytime AP was 58.60 (REAL), 57.55 (SYNTH), 66.77 (MIX), 64.73 (SAMPLER), and 67.50 (CARE). The authors report MIX at +8.17% and CARE at +8.90% relative to REAL, and note the synthetic-only model suffered from adding the nighttime condition, while SAMPLER on average attained results similar to before.
• Real-only models collapse at night. Nighttime AP was 15.21 for REAL, 43.22 for SYNTH, 43.38 for MIX, 42.00 for SAMPLER, and 43.75 for CARE. The REAL model showed a drop of -43.39% and predicted no bounding box on the example nighttime image in Figure 2, while all models trained with synthetic data detected the runway.
• Synthetic data preserves nighttime performance. The authors report that the non-REAL methods maintained an AP score similar to the SYNTH model, with a -14.33% drop compared to the daytime validation set, which is far better than REAL on both validation sets.
• The customized CARE loss is best overall. CARE achieved the best score in both the daytime and nighttime evaluations, though by a small margin at night. The authors state that the CARE sampler alone appears to lower detection performance, but the alignment loss compensates for that drop and improves final results when the training data has enough diversity.
• Real data alone is not enough. The conclusion states that synthetic images can improve model accuracy, but the presence of both real and synthetic images is necessary.

Methodology in Plain English

Real images came from an aircraft equipped with cameras and sensors photographing runways during landing sequences at airports in the U.S., covering 27 airports with associated metadata. Images from the same airport tend to look alike, so diversity is limited. Labels were produced automatically from aircraft GPS positions and runway positions, which can introduce errors from miscalibration or cloud occlusions. For training, the authors used 1,000 real images from 19 airports.

Synthetic images were generated with the commercial flight simulator XPlane12, where the aircraft's position, orientation, weather and time can be specified. This produced a dataset of 10,000 images from 199 airports, including the airports present in the real dataset, with automatic and precise bounding-box labels. A custom pipeline was used instead of existing datasets to obtain a large volume (more than 5,000) of high-quality images. For the nighttime study, a second synthetic dataset was generated with 5,000 daytime images and 5,000 nighttime images from the same 199 airports.

Five training strategies were compared, all keeping the same number of images per airport to avoid imbalance: REAL (real images only), SYNTH (synthetic only), MIX (random sampling from the combined dataset, giving a variable synthetic-to-real ratio per step), SAMPLER (the CARE sampling scheme, which forces an equal number of synthetic and real images in each minibatch, used without the alignment loss), and CARE (the full method). The detector was a Faster R-CNN with FPN and a ResNet50 backbone pretrained on COCO2017, trained with batch size 8, learning rate 0.002, 10,000 iterations, and a balancing factor of 0.1 for the CARE alignment loss, which minimizes the Euclidean distance between feature maps of real and synthetic objects.

Evaluation used COCO Average Precision (AP), averaged over Intersection over Union thresholds from 0.5 to 0.95 in 0.05 steps. Testing used 200 real daytime images from 8 airports that appear in no training set, plus 200 real nighttime images from the same 8 airports; real nighttime images were never seen during training.

Why This Matters

Impact on research: The paper provides a concrete, quantitative case study of sim-to-real transfer in a safety-critical aviation task where public real data are scarce, and shows that a domain adaptation method can be simplified for a single-class detection problem without losing its benefit. It also adds nighttime runway detection to the small body of work on synthetic data for runway recognition, alongside datasets such as LARD, FS2020 and Rareplanes discussed in the related work.

Real-world applications:

• Vision-based landing systems on commercial aircraft that must position the aircraft during approach.
• Augmenting scarce or restricted real imagery with simulator data for certification-adjacent perception models.
• Training detectors for rare operating conditions such as nighttime operations, where real examples are difficult or unsafe to collect at scale.
• Airport and airline ground operations, which the authors identify as an area where safety and routing optimization could benefit from deep vision models.

Industry relevance: The work is a collaboration between the Fédération ENAC ISAE-SUPAERO ONERA (Université de Toulouse) and Airbus, and the disclosure notes that three authors are Airbus employees. It addresses a practical industrial constraint: data collection and labeling costs are too high for a single company or product, and flight simulators are cheap, customizable, and generate precise labels automatically.

Future Directions

• Extend the approach to other adverse conditions, such as snowy or foggy weather, which are harder to capture in reality or render in simulation.
• Investigate the impact of the CARE sampler, which appeared to lower detection performance on its own, to further improve the domain adaptation benefits.
• Address the limitations of the real data noted in the paper, including redundancy across images from the same airports and label errors from miscalibration or cloud occlusions.
• Build on the finding that both real and synthetic data are necessary, to determine how much real data is needed alongside large synthetic datasets.

Target Audience

Researchers and engineers working on object detection, synthetic data generation, and domain adaptation, particularly those in aviation and other safety-critical industries where real labeled data are scarce or costly. It is also relevant to practitioners evaluating commercial flight simulators as a data source, and to readers interested in how perception models behave on rare conditions such as nighttime runway detection.

Authors’ abstract

Deep vision models are now mature enough to be integrated in industrial and possibly critical applications such as autonomous navigation. Yet, data collection and labeling to train such models requires too much efforts and costs for a single company or product. This drawback is more significant in critical applications, where training data must include all possible conditions including rare scenarios. In this perspective, generating synthetic images is an appealing solution, since it allows a cheap yet reliable covering of all the conditions and environments, if the impact of the synthetic-to-real distribution shift is mitigated. In this article, we consider the case of runway detection that is a critical part in autonomous landing systems developed by aircraft manufacturers. We propose an image generation approach based on a commercial flight simulator that complements a few annotated real images. By controlling the image generation and the integration of real and synthetic data, we show that standard object detection models can achieve accurate prediction. We also evaluate their robustness with respect to adverse conditions, in our case nighttime images, that were not represented in the real data, and show the interest of using a customized domain adaptation strategy.