Small Object Detection for Birds with Swin Transformer

AI summary

Overview

Research area: Computer vision — object detection, specifically small object detection and transformer-based detection architectures (Swin Transformer, CenterNet).

Technical level: Intermediate. Readers should be familiar with standard detection pipelines (backbone, neck, prediction head), anchor-based versus anchor-free detectors, and the general idea of window-based self-attention.

Scope: The paper proposes a Swin Transformer-based neck network for detecting small, sparse bird objects, and studies how the attention window size in that neck affects detection accuracy.

What This Paper Is About

Detecting small objects (the paper defines these as objects under 32 pixels) is much harder than detecting normally sized objects, because object features vanish through repeated down-sampling and up-sampling, and the standard receptive field does not match small object size. Existing small object methods are built for crowded, dense scenes such as pedestrians in a crowd or distant objects in remote sensing imagery, but when objects are both small and sparse there are not enough instances to learn effective features. This paper targets one such sparse small object category — birds — and redesigns the neck network between the backbone and the prediction head to learn more effective features for them.

Key Contributions

1. A hierarchical neck network based on Swin Transformer rather than a CNN neck (CenterNetNeck or FPN), built on an anchor-free CenterNet baseline.
2. A change to the default shifted window size in the neck, making it smaller to focus attention on small objects and to allow cross-window attention to be computed around them.
3. An "Up Merging" upsampling module for the neck, which is the reverse of the Swin Transformer Patch Merging down-sampling operation and is used instead of transposed convolution.
4. An experimental demonstration, on two bird datasets, that the Swin Transformer neck and the reduced window size both improve average precision.

Main Findings

• Swin Transformer neck beats the CNN neck: On the MVA2023 validation set, the proposed Swin Transformer neck reached AP50 0.898, AP75 0.372, and AP 0.745, compared with CenterNet neck results of AP50 0.846, AP75 0.337, and AP 0.702.
• Smaller windows are better across all metrics: Reported before hard-negative training on the MVA2023 validation set, window size 2 gave AP50 0.702, AP75 0.171, AP 0.549; window size 3 gave AP50 0.693, AP75 0.166, AP 0.538; window size 5 gave AP50 0.684, AP75 0.158, AP 0.53.
• Window size 2 is best specifically for small objects: For objects under 32 pixels (AP_S), the paper reports that smaller window sizes, especially size 2, yield better AP_S. The AP_S values are presented in a figure using the Drone2021 validation set and are not given as numbers in the text.
• Challenge leaderboard result: After training, fine-tuning, and hard-negative training, the method obtained AP50 of 0.702 on the online public test containing 9,699 images on the MVA challenge server.
• Rationale for the window change: With a default window size of 7, a bird smaller than 32 pixels falls inside a single window and cross-window attention is never computed for it; reducing the window size lets the model attend to surrounding context (the paper's illustration uses sky and cloud around a bird).

Methodology in Plain English

The authors start from CenterNet, an anchor-free detector that predicts object center points, box width and height, and a center offset, rather than relying on pre-defined anchor boxes. They replace the ResNet-50 backbone with Swin-S, a Swin Transformer pre-trained on ImageNet22K, so the model produces multiple levels of features.

Between the backbone and the prediction head, they build a new neck. Each stage of this neck contains multiple Swin Transformer blocks (two by default). The final backbone output feeds the first neck stage, and, as in the standard Swin design, self-attention is computed inside local windows, with the window partitioning shifted between consecutive layers so that attention crosses the previous window boundaries. The key change is that the window is made much smaller than the default 7, so that a small bird is not swallowed inside one large window and its surrounding context can be attended to.

Between neck stages, features are upsampled by an "Up Merging" module — the reverse of Patch Merging, also seen in PixelShuffle-style super-resolution — with a stride of 2, which reduces the channel count from 4C to C, followed by a linear layer that changes C to 2C to match the backbone features. A U-Net-like set of skip connections concatenates backbone features (C2, C3, C4) with neck features, to counter the information lost over the long down- and up-sampling path.

Training proceeds in stages: 150 epochs on Drone2021 with a learning rate of 5e-6, fine-tuning for 100 epochs on MVA2023 with a learning rate of 1e-5, then hard-negative training on MVA2023 with a hard-negative rate of 0.3. The CenterNet head's center-point loss gamma is set to 6.0 and the bounding box L1 loss weight to 0.2. The two evaluation datasets are Drone2021 (47,260 images, 60,971 annotated bird instances) and MVA2023 (9,759 images, 29,037 annotated bird instances).

Why This Matters

Impact on research: The work shows that the neck — often an afterthought compared with the backbone and head — is a meaningful lever for small object detection, and that a hyperparameter inherited from general-purpose vision (the window size of 7) is not optimal when the target objects are small. It also gives a concrete alternative for the sparse-small-object regime, which the paper notes has been under-served by methods designed for dense small-object scenes.

Real-world applications:

• Drone-based bird surveys and wildlife monitoring, the setting of the Drone2021 and MVA2023 datasets used here.
• Ecological and conservation monitoring, where counting and locating individual birds over large areas matters.
• Aviation safety, where detecting birds near aircraft or airfields is directly relevant.
• Remote sensing and surveillance more broadly, which the paper lists among the general applications of object detection.

Industry relevance: The system builds on an anchor-free, end-to-end detector and reports a leaderboard result on an online public challenge test server, which places it in a practically benchmarked setting rather than only an offline evaluation. The Up Merging module is a lightweight upsampling alternative to transposed convolution, relevant to efficiency-sensitive deployments.

Future Directions

• Reporting AP_S numerically for all window sizes and on more datasets; in this paper the small-object AP values appear only in a figure for the Drone2021 validation set.
• Testing whether even smaller windows than 2, or window sizes adapted per stage, further improve small object accuracy.
• Extending the Swin Transformer neck to other sparse small object categories beyond birds, since the paper's motivation about sparse targets is not bird-specific.
• Comparing the Up Merging module against transposed convolution under matched conditions, and examining how much the U-Net-style skip connections contribute independently of the neck design.

Target Audience

Researchers and practitioners working on small object detection, transformer-based detection architectures, or aerial and drone-based wildlife imagery. It is most useful to readers who already understand standard detection pipelines and want a focused, incremental design study on the neck and attention window size, and to engineers building bird or similarly small-object detectors who need benchmarked reference numbers.

Authors’ abstract

Object detection is the task of detecting objects in an image. In this task, the detection of small objects is particularly difficult. Other than the small size, it is also accompanied by difficulties due to blur, occlusion, and so on. Current small object detection methods are tailored to small and dense situations, such as pedestrians in a crowd or far objects in remote sensing scenarios. However, when the target object is small and sparse, there is a lack of objects available for training, making it more difficult to learn effective features. In this paper, we propose a specialized method for detecting a specific category of small objects; birds. Particularly, we improve the features learned by the neck; the sub-network between the backbone and the prediction head, to learn more effective features with a hierarchical design. We employ Swin Transformer to upsample the image features. Moreover, we change the shifted window size for adapting to small objects. Experiments show that the proposed Swin Transformer-based neck combined with CenterNet can lead to good performance by changing the window sizes. We further find that smaller window sizes (default 2) benefit mAPs for small object detection.