BurstDeflicker: A Benchmark Dataset for Flicker Removal in Dynamic Scenes

AI summary

Overview

• Research area: Computer vision / image restoration — specifically multi-frame flicker removal (MFFR) from short-exposure images captured under AC-powered lighting, and the construction of a paired benchmark dataset for that task.
• Technical level: Advanced. The paper assumes familiarity with Retinex theory, rolling-shutter camera behavior, burst/multi-frame image restoration architectures, and full-reference and no-reference image quality metrics.
• Scope: This paper introduces BurstDeflicker, the first dataset for multi-frame flicker removal, combining an unlimited synthetic subset, 4,000 real-world static flickering image pairs, and 3,690 green-screen composited dynamic image pairs, and benchmarks several restoration networks on it.

What This Paper Is About

Flicker artifacts appear as dark bands and uneven brightness in short-exposure photos because AC-powered lights vary in intensity over time while rolling-shutter cameras expose each sensor row at a slightly different moment. These artifacts hurt both visual quality and downstream tasks like detection and tracking, yet no large-scale, realistic paired dataset existed for training and evaluating removal models — largely because dynamic scenes cannot be captured twice with identical motion. The paper builds such a dataset from three complementary sources and validates it with quantitative and qualitative experiments.

Key Contributions

1. BurstDeflicker, the first MFFR dataset. It combines an unlimited number of synthetic images, 4,000 real-world captured flickering image pairs (BurstDeflicker-S), and 3,690 manually constructed dynamic image pairs (BurstDeflicker-G).
2. A Retinex-based flicker synthesis method that jointly models ambient and flickering illumination, supporting diverse flicker patterns from different light-source rectification modes (full-wave, half-wave, and PWM), with controllable flicker intensity, area, and frequency.
3. A green-screen compositing method that overlays foreground subjects with motion onto real flickering backgrounds, producing dynamic paired data that preserves real flicker degradation and helps mitigate motion ghosting in multi-frame restoration.
4. A redefinition of the flicker removal target: rather than completely eliminating flicker-induced illumination (as prior work did by modeling the clean image as reflectance times ambient light only), the paper argues the flicker illumination should be adjusted to its effective value.

Main Findings

• Retinexformer fails out of the box: the pre-trained low-light enhancement model achieves PSNR 15.704, SSIM 0.707, LPIPS 0.213 on static test data (MUSIQ 53.596, PIQE 50.269, BRISQUE 30.242 on dynamic data), showing that training on a low-light dataset alone does not address flicker.
• Retraining on BurstDeflicker helps prior methods: Lin et al.'s network improves from PSNR 20.358 / SSIM 0.838 / LPIPS 0.134 to PSNR 26.408 / SSIM 0.875 / LPIPS 0.102; Retinexformer improves to PSNR 27.212 / SSIM 0.885 / LPIPS 0.081.
• Restormer performs best among the three-frame baselines: PSNR 30.634, SSIM 0.918, LPIPS 0.045 on static data and MUSIQ 59.097, PIQE 34.896, BRISQUE 19.324 on dynamic data, ahead of HDRTransformer (30.031 / 0.914 / 0.054) and Burstormer (29.439 / 0.910 / 0.056).
• Synthetic pre-training reduces overfitting: training only on synthetic data gives PSNR 24.483 / SSIM 0.862 / LPIPS 0.122 and dynamic MUSIQ 57.096, clearly below combinations that include real data.
• Green-screen data specifically improves dynamic performance: the setting that includes BurstDeflicker-G reaches MUSIQ 59.097, PIQE 34.896, BRISQUE 19.324 on dynamic test data, and its absence produces motion ghosting artifacts in the visual comparison.
• More input frames help, with diminishing returns: PSNR rises from 27.310 (single image) to 30.264 for Burst-2 (+2.594) and 30.634 for Burst-3 (+3.324). The gain from 2 to 3 frames (0.730 dB) is smaller than from 1 to 2 frames.
• Qualitative behavior differs by method: Retinexformer globally brightens images and introduces color shifts; DeflickerCycleGAN (Lin et al.) has little effect under severe flicker and insufficient lighting; Restormer trained on BurstDeflicker handles both mild indoor flicker and strong nighttime flicker.
• Stated limitations: when an AC-powered light source is the sole illumination, flicker degradation can be severe enough to cause color shifts in restoration; multi-frame methods struggle when frames lack complete scene content; and strong handheld jitter misalignment can degrade multi-frame performance to that of single-frame methods.

Methodology in Plain English

The authors attack the data problem from three angles rather than trying to capture everything in one way.

Synthetic data via Retinex theory. They treat a scene's light as the sum of a flickering component and a stable ambient component, with image appearance being reflectance multiplied by illumination. Crucially, they define the "clean" target as reflectance times the sum of ambient light plus the effective (average) value of the flickering light, not as reflectance times ambient light alone. From this they derive an expression relating a flickering image to a clean image, controlled by the ratio between ambient and flicker intensity (set from 0 to 1) plus the flicker pattern. Three rectification modes are modeled — full-wave, half-wave, and PWM — each producing a different flicker signature based on electric network frequency (50 or 60 Hz), the camera's row scanning frequency (100 kHz to 160 kHz for 512×512 resolution), and an initial phase. Backgrounds come from the indoorCVPR dataset, and each burst keeps the same pattern while only the AC phase changes. This subset is unlimited in size and used to pre-train networks.

Real captures for realism. Using a Canon EOS R7 with a 15–150mm f/2.8 lens and a Canon EOS R6 Mark II with a 24–105mm f/4–7.1 STM lens on tripods, with remote shutter release and electronic shutter burst mode, they capture 10 consecutive frames per burst. Flickering frames use 1/1000–1/2000 second exposures; the clean reference uses 1/50 or 1/60 second (depending on the local electric network frequency), integrating multiple flicker cycles. ISO is adjusted so both receive the same light. They cover 369 real-world scenes, both indoor (offices, supermarkets, subway stations) and outdoor (LED billboards, parking lots), yielding 4,000 images.

Green-screen compositing for motion. To simulate motion, they take real flickering pairs as backgrounds and composite green-screen foregrounds from the VideoMatte240K dataset using alpha masks, with foregrounds manually selected for semantic and spatial compatibility and composited in Adobe After Effects. Because the clean ground truth is a single frame, it is replicated ten times and the same ten foreground clips are overlaid. This produces 3,690 dynamic images.

Training and evaluation setup. Static data is split 8/2 into train and test. Synthetic camera shake is applied via random rotations in [-3°, 3°] and translations in [-5, 5] pixels. Images are resized rather than cropped during training, because flicker is localized and periodic along the line-scan direction and cropping would break that periodicity. Each iteration samples three frames at intervals of 1 to 3 and concatenates them as input, with a single clean image as target. Because collecting paired MFFR data is expensive, models are pre-trained on synthetic data and then fine-tuned on BurstDeflicker. Evaluation uses PSNR, SSIM, and LPIPS on static data, and MUSIQ, PIQE, and BRISQUE on 50 dynamic sequences captured with mobile devices and consumer cameras (no ground truth is available for those). Restormer was reduced (refinement blocks from 4 to 2, feature channels 32 instead of 48) for the 24 GB RTX 3090, and Burstormer was modified from 8-frame to 3-frame input for consistency.

Why This Matters

Impact on research. The paper addresses a genuine dataset gap: the authors state that no publicly available dataset specifically designed for flicker removal existed, which is a major obstacle for training and evaluating deep models. Prior synthetic approaches (Wong et al., adapted by Lin et al.) were designed for geo-tagging and modeled flicker removal as total elimination of flicker illumination, which the authors argue lacks variability and realism and leads to poor generalization. By redefining the target and supplying real and motion-containing paired data, BurstDeflicker provides a benchmark and a training resource for a task that had none.

Real-world applications.

• High-speed photography, where short exposures are required and flicker is most visible.
• HDR imaging, which relies on multiple exposures including short ones.
• Slow-motion video recording, where short exposures are frequently used.
• Scenes with abundant artificial lighting such as amusement parks, lantern festivals, and cinemas, where flicker can ruin valuable moments for users.
• Downstream computer vision tasks such as object detection, tracking, and recognition, whose reliability depends on consistent lighting.

Industry relevance. The work stems from a collaboration spanning Nankai University, Peng Cheng Laboratory, The Hong Kong Polytechnic University, and OPPO Research Institute, and is partly funded by the OPPO Research Fund — a strong signal of direct smartphone-camera relevance. The dataset targets consumer-grade cameras that use rolling shutter and line-scan exposure, and the paper explicitly notes that hardware-level solutions such as flicker detection circuits are costly, hard to access, and complex to calibrate, which limits deployment in consumer-level applications. A software, data-driven alternative is therefore attractive to device manufacturers.

Future Directions

• Handling extreme flicker and color shifts. The authors note that when an AC-powered light source is the sole illumination, restoration can produce noticeable color shifts, and suggest designing network architectures tailored to flicker's specific priors.
• Improving multi-frame restoration under incomplete content. Performance still suffers when available frames lack complete scene content, leaving room for methods that reason about missing information across a burst.
• Robustness to large inter-frame misalignment. Under strong handheld jitter, multi-frame restoration can degrade to single-frame performance, motivating better alignment or motion-handling strategies.
• Extending dynamic paired data. The green-screen method is a semi-synthetic workaround for the non-repeatability of dynamic scenes; further strategies for capturing or synthesizing realistic dynamic flicker pairs remain open.

Target Audience

Researchers and engineers working on image restoration, computational photography, and mobile camera pipelines — particularly those interested in multi-frame or burst-based methods, paired dataset construction for hard-to-capture degradations, and the interaction between rolling-shutter sensors and artificial lighting. It is also relevant to practitioners building detection, tracking, or recognition systems that operate under AC-powered lighting, and to camera hardware and imaging teams at consumer device companies. Readers without background in Retinex decomposition, burst restoration architectures, or image quality metrics will find the experimental sections difficult, so the paper is best suited to those already working in computational imaging.

Authors’ abstract

Flicker artifacts in short-exposure images are caused by the interplay between the row-wise exposure mechanism of rolling shutter cameras and the temporal intensity variations of alternating current (AC)-powered lighting. These artifacts typically appear as uneven brightness distribution across the image, forming noticeable dark bands. Beyond compromising image quality, this structured noise also affects high-level tasks, such as object detection and tracking, where reliable lighting is crucial. Despite the prevalence of flicker, the lack of a large-scale, realistic dataset has been a significant barrier to advancing research in flicker removal. To address this issue, we present BurstDeflicker, a scalable benchmark constructed using three complementary data acquisition strategies. First, we develop a Retinex-based synthesis pipeline that redefines the goal of flicker removal and enables controllable manipulation of key flicker-related attributes (e.g., intensity, area, and frequency), thereby facilitating the generation of diverse flicker patterns. Second, we capture 4,000 real-world flicker images from different scenes, which help the model better understand the spatial and temporal characteristics of real flicker artifacts and generalize more effectively to wild scenarios. Finally, due to the non-repeatable nature of dynamic scenes, we propose a green-screen method to incorporate motion into image pairs while preserving real flicker degradation. Comprehensive experiments demonstrate the effectiveness of our dataset and its potential to advance research in flicker removal.