SPWOOD: Sparse Partial Weakly-Supervised Oriented Object Detection

AI summary

Overview

• Research area: Computer vision, specifically oriented object detection (OOD) in remote sensing imagery, at the intersection of semi-supervised learning, weakly supervised learning, and sparsely supervised learning.
• Technical level: Advanced. The paper assumes familiarity with rotated bounding box detection, teacher-student pseudo-labeling pipelines, Focal Loss, Feature Pyramid Networks, and Gaussian Mixture Models.
• Scope: The paper proposes a single unified training framework (SPWOOD) that learns from a few sparse, weakly annotated images (RBox, HBox, Point, or combinations) plus a large pool of unlabeled images, and validates it on DOTA-v1.0, DOTA-v1.5, and DIOR.

What This Paper Is About

Oriented object detection in remote sensing is expensive to annotate because objects are densely packed and numerous. Existing methods reduce this cost by dropping either the amount of annotation (sparse, where only some objects in an image are labeled) or its detail (weak, where only HBoxes or points are given), but prior work has not combined both reductions in one framework. SPWOOD is proposed as the first Sparse Partial Weakly-Supervised Oriented Object Detection framework, designed to train well from a few sparse weakly-labeled images plus many unlabeled images, and to accept RBox, HBox, Point, or mixed annotations in the same pipeline.

Key Contributions

1. The first SPWOOD framework, a unified training pipeline that robustly supports multi-format annotations (RBox, HBox, Point, or their combination) as input under sparse partial annotation settings.
2. The Sparse-annotation-Orientation-and-Scale-aware Student (SOS-Student) model, which learns to separate unlabeled objects from background in sparsely labeled settings and to recover orientation and scale information from orientation-agnostic or scale-agnostic weak annotations.
3. The Multi-level Pseudo-label Filtering (MPF) mechanism, which models the teacher's prediction-confidence distribution separately per FPN level (P3, P4, P5, P6, P7) using a Gaussian Mixture Model, instead of relying on a single class-agnostic threshold.
4. The Overall Sparse Method for constructing sparse datasets, which treats all labeled annotations across the dataset as one unified group so that a consistent sampling ratio is applied to every category, keeping the sparse dataset's distribution aligned with the original dataset.

Main Findings

• DOTA-v1.0 gains over the SAOD baseline (RSST*): Under RBox annotation, SPWOOD reaches 48.5, 54.0, 54.9, 57.8, 54.9, and 60.3 mAP at the 10-10, 10-20, 20-10, 20-20, 30-10, and 30-20 sparse-partial ratios, versus 42.4, 45.5, 41.7, 51.0, 53.2, and 56.5 for RSST*.
• HBox annotations beat RBox baselines: SPWOOD with only HBox annotations (45.5, 51.9, 52.2, 54.0, 53.1, 56.5) outperforms RSST* using corresponding proportions of RBox annotations by 3.1% (10-10), 6.4% (10-20), 10.5% (20-10), and 3.0% (20-20).
• Large margins over weakly supervised methods: Against H2RBox-v2 (HBox) SPWOOD gains 14.9% (10-10), 21.1% (10-20), 13.7% (20-10), 11.3% (20-20), 9.2% (30-10), and 7.3% (30-20). Against Point2RBox-v2 (Point) it improves mAP by 17.9%, 27.0%, 19.1%, 15.6%, 20.7%, and 14.8% at the same ratios.
• Mixed weak annotations work: At the 20-20 ratio, an RBox:HBox:Point mix of 1:1:1 gives 53.0 mAP, 1:1:0 gives 56.3, 1:0:1 gives 52.3, 0:1:1 gives 47.6, and 0:1:4 gives 41.2.
• DOTA-v1.5 results: SPWOOD scores 43.2, 47.9, 49.0, 52.1, 51.3, and 53.1 mAP at 10-10, 10-20, 20-10, 20-20, 30-10, 30-20, beating RSST* (36.2, 35.8, 42.3, 46.0, 45.7, 49.8) by 7%, 12.1%, 6.7%, 6.1%, 5.6%, and 3.3% respectively.
• DIOR results: SPWOOD obtains 44.1 (10-20), 45.7 (20-20), and 46.3 (30-20) mAP, ahead of RSST (40.7, 44.8, 46.1), RSST* (38.8, 43.2, 45.7), S²Teacher (36.1, 42.6, 45.1), PWOOD (31.0, 36.8, 39.1), and MCL (30.8, 33.1, 35.8).
• Very sparse ratio 20-50: SPWOOD reaches 63.0 mAP versus PWOOD 59.8, S²Teacher 56.5, RSST 56.1, RSST* 55.1, and MCL 53.2.
• MPF beats class-agnostic filtering (CPF): MPF gives 49.5, 54.0, 54.9, 57.8 mAP at 10-10, 10-20, 20-10, 20-20 versus CPF's 44.4, 53.0, 51.9, 57.1 — improvements of 5.1%, 1.0%, 3.0%, and 0.7%, with the largest gain in the most sparse setting.
• The Single Sparse Method actually outperforms the proposed Overall Sparse Method in the reported numbers: Single Sparse scores 49.5, 54.0, 54.9, 57.8, 54.9, 60.3 versus Overall Sparse's 41.7, 49.1, 47.9, 57.2, 52.3, 57.5, a margin of 7.8%, 4.9%, 7.0%, 0.6%, 2.6%, and 2.8% in favor of Single Sparse. The analysis attributes this to Single Sparse retaining more annotations for initially scarce categories; for baseball diamond at ratio 10-10, Single Sparse produced a 164.3% relative increase in annotation count, which translated into a 98.6% relative AP difference.
• Sparse Annotation Learning (SAL) module matters: On DOTA-v1.0 at 20-20, the framework scores 47.6 mAP with SAL and 42.0 without it. A weight study on the suppression factor reports 57.8 (0.4), 57.5 (0.3), 60.6 (0.2), and 58.0 (0.1).
• Higher compute cost: On DIOR, SPWOOD uses 22785MB and 40 hours, compared with RSST (13672MB, 34 hours), PWOOD (9021MB, 23 hours), and MCL (5598MB, 16 hours). The paper attributes this to handling multiple weak annotation formats and to compensating for highly sparse supervision.
• Stated limitation: The approach uses only a single visual modality; the authors suggest adding spectral or textual information as future work.

Methodology in Plain English

The framework follows the classic teacher-student, pseudo-labeling recipe for semi-supervised detection, split into two stages.

Stage 1 — Burn-in. A student detector is trained on the small set of sparse, weakly annotated images, using both the original and an augmented view of each image. Its weights are copied to a teacher model.

Stage 2 — Self-training. Large amounts of strongly and weakly augmented unlabeled data are passed through the teacher, which produces pseudo-labels. The student is then trained on these pseudo-labels, and the teacher is updated as an exponential moving average (EMA) of the student's weights.

Three pieces make this work under sparse, weak supervision:

• Handling unlabeled objects. In sparse annotation, unlabeled objects and true background both get the "background" label, so the model is wrongly penalized for detecting real objects. The student's classification loss is split into three cases — labeled objects, background, and "hard negatives" (high-confidence predictions matched to background). The hard-negative case gets multiplied by an adaptive factor ω that down-weights this misleading signal, inheriting the robustness of Focal Loss while targeting sparse scenarios.
• Recovering orientation without angle labels. Images are augmented by flipping or rotating, producing views whose predicted angles must obey a known relationship. A Smooth-L1 angle loss enforces these relationships, so orientation is learned from HBox-level annotations.
• Recovering scale without box labels. The model uses a Gaussian overlap loss (Bhattacharyya distance between predicted boxes modeled as 2D Gaussians) to bound object scale from above, and a Voronoi Watershed loss to bound it from below: Voronoi regions isolate each point annotation, a watershed algorithm gives pixel-level segmentation, and the result is rotated to match the predicted box direction to regress width and height.
• Filtering pseudo-labels. Instead of one static threshold or one class-agnostic model over all predictions, each FPN level's confidence scores are fitted with a two-component Gaussian Mixture Model (positive and negative distributions), initialized with means at the max/min predicted scores, variances at 1, and weights at 0.5. Expectation-Maximization then finds the per-level threshold τ.

Training uses MMRotate with an FCOS detector, ResNet50 backbone, and FPN neck, optimized with AdamW for 180,000 iterations including a 12,800-iteration burn-in. Loss weights w_cls, w_cen, w_box are set to 1, and (w_Ang, w_O, w_W) to (0.2, 10, 5).

Sparse datasets are built by selecting 10%, 20%, or 30% of images as labeled and treating the rest as unlabeled, then applying a sparse ratio (10%, 20%, 30%) to annotations. Naming is "partial-ratio–sparse-ratio," so 30-10 means 30% of images labeled with 10% of annotations kept. Weak annotations are created by dropping orientation and scale information.

Why This Matters

• Impact on research: The paper argues that annotation cost can be reduced along two axes at once — fewer labeled images and less detailed labels — rather than one at a time, and provides a single pipeline that accepts RBox, HBox, Point, or mixtures. It also questions a common assumption in sparse-dataset construction by showing that the widely used Single Sparse Method empirically outperformed the proposed Overall Sparse Method on the reported benchmarks, and analyzes why (rare categories get disproportionately retained).
• Real-world applications:
  • Aerial and satellite image analysis for detecting vehicles, ships, planes, and storage tanks with far fewer annotation hours.
  • Rapid mapping and disaster response, where imagery is abundant but expert annotators are scarce and speed matters.
  • Agricultural and land-use monitoring, where categories are numerous and per-image labeling is costly.
  • Infrastructure inspection and urban planning, where oriented objects (buildings, roads, equipment) need rotated boxes but only coarse labels may be available.
• Industry relevance: Remote sensing and geospatial companies pay heavily for rotated-box annotation. A framework that reaches 63.0 mAP at the 20-50 ratio and beats the same baseline using only HBox labels offers a direct cost lever, at the price of higher memory (22785MB) and runtime (40 hours) reported on DIOR, which matters for deployment planning.

Future Directions

• Multi-modal input. The authors state the current limitation is single-modality vision and suggest incorporating spectral or textual information to improve performance under sparse annotation.
• Resolving the sparse-sampling contradiction. Since Single Sparse Method outperformed Overall Sparse Method in the reported experiments while Overall Sparse is presented as the fairer, distribution-preserving approach, follow-up work could investigate when distribution preservation helps rather than hurts.
• Reducing computational cost. The framework is the most expensive of the compared methods; making the multi-format and sparse-compensation overhead cheaper is an open engineering problem.
• Failure modes under intra-class variance. The failure analysis figure shows cases where the model misses objects due to intra-class variance (with the full analysis truncated in the provided content), which points to a remaining robustness gap.
• Extending to the truncated/omitted hard cases. The provided content shows only the first scenarios in the failure analysis (effectiveness in sparse settings and the intra-class variance challenge) before cutting off, so further failure categories are not reported here.

Target Audience

Researchers and engineers working on remote sensing object detection, semi-supervised and weakly supervised learning, and rotated bounding box detection, particularly those who need to cut annotation budgets while keeping detection accuracy comparable to fully supervised OOD algorithms. It is also relevant to practitioners in geospatial analytics who want a single pipeline that tolerates mixed label formats, and to readers interested in how pseudo-label filtering and dataset sparsification strategies interact.

Authors’ abstract

A consistent trend throughout the research of oriented object detection has been the pursuit of maintaining comparable performance with fewer and weaker annotations. This is particularly crucial in the remote sensing domain, where the dense object distribution and a wide variety of categories contribute to prohibitively high costs. Based on the supervision level, existing oriented object detection algorithms can be broadly grouped into fully supervised, semi-supervised, and weakly supervised methods. Within the scope of this work, we further categorize them to include sparsely supervised and partially weakly-supervised methods. To address the challenges of large-scale labeling, we introduce the first Sparse Partial Weakly-Supervised Oriented Object Detection framework, designed to efficiently leverage only a few sparse weakly-labeled data and plenty of unlabeled data. Our framework incorporates three key innovations: (1) We design a Sparse-annotation-Orientation-and-Scale-aware Student (SOS-Student) model to separate unlabeled objects from the background in a sparsely-labeled setting, and learn orientation and scale information from orientation-agnostic or scale-agnostic weak annotations. (2) We construct a novel Multi-level Pseudo-label Filtering strategy that leverages the distribution of model predictions, which is informed by the model's multi-layer predictions. (3) We propose a unique sparse partitioning approach, ensuring equal treatment for each category. Extensive experiments on the DOTA and DIOR datasets show that our framework achieves a significant performance gain over traditional oriented object detection methods mentioned above, offering a highly cost-effective solution. Our code is publicly available at https://github.com/VisionXLab/SPWOOD.