HyperCOD: The First Challenging Benchmark and Baseline for Hyperspectral Camouflaged Object Detection

AI summary

Overview

Research area: Computer vision, specifically hyperspectral image analysis, camouflaged object detection, and adaptation of segmentation foundation models (SAM) to non-RGB modalities.

Technical level: Advanced. The paper assumes familiarity with transformer-based segmentation, spectral-spatial feature decomposition, token pruning, and standard COD evaluation metrics (MAE, E-measure, S-measure, adaptive F-measure).

Scope: The paper introduces HyperCOD, a 350-image hyperspectral camouflaged object detection benchmark, and HSC-SAM, a spectral-aware adaptation of SAM that sets the reported state of the art on it.

What This Paper Is About

Camouflaged object detection tries to find objects that blend into their surroundings. Existing methods work on ordinary RGB photos, which fail when an object matches its background in color and texture. Hyperspectral cameras capture hundreds of narrow spectral bands and can reveal material differences that RGB cannot, but no large-scale benchmark existed for hyperspectral camouflaged object detection (HCOD). This paper builds that benchmark (HyperCOD) and a baseline model (HSC-SAM) that adapts SAM to hyperspectral inputs.

Key Contributions

1. HyperCOD dataset: the first large-scale, challenging benchmark for hyperspectral camouflaged object detection, with 350 high-resolution hyperspectral cubes captured across diverse scenes and five camouflage challenge categories.
2. HSC-SAM framework: a unified spectral-aware detection framework that integrates spectral-spatial decomposition with complementary prompt learning to adapt SAM to the hyperspectral domain.
3. Spectral-Guided Token Dropout (SGTD) and Fusion Detail Enhancer / Fine-Grained Detail Enhancement (FDE): modules for saliency-guided token pruning and boundary refinement, aimed at precise segmentation and reduced computation.
4. Empirical validation: extensive comparison against RGB-based COD, hyperspectral salient object detection, and SAM-based methods, plus ablations of each component and of the SGTD threshold, along with a stated generalization to other public HSI datasets.

Main Findings

• Dataset scale and content: HyperCOD contains 350 high-quality hyperspectral images, each with 200 spectral bands from 400–1000 nm at a spatial resolution of 1240 × 1680 pixels (listed as 1680 × 1240 in Table 1). It is split into 280 training and 70 testing samples at a 4:1 ratio, captured across 11 diverse natural environments.
• Five challenge attributes: Minimal Objects (extremely small targets), Complex Shapes (irregular or fragmented boundaries), Dynamic Lighting (strong illumination variation), Object Occlusion, and Cluttered Backgrounds. Samples with object-to-image area ratios below 0.5% are labeled tiny-object cases; samples with edge-to-perimeter ratios exceeding 0.3 are labeled complex-edge cases.
• State-of-the-art headline results: HSC-SAM reaches MAE 0.0017, E 0.853, S 0.802, and Adp-F 0.681 on HyperCOD. Its MAE is 0.0005 lower than the second-best (SAM2-UNet at 0.0022), and its Adp-F of 0.681 is a reported 3.1% improvement over the closest competitor, Camoformer (0.660).
• Metric-level nuance in Table 2: Although the paper states state-of-the-art results "across all key metrics," the table shows SAM2-UNet reporting higher E (0.899 vs. 0.853) and higher S (0.805 vs. 0.802) than HSC-SAM. HSC-SAM leads on MAE and Adp-F.
• Efficiency: HSC-SAM uses 11.7M parameters and 94.2G FLOPs, compared with SAM2-UNet at 216.4M parameters and 128.4G FLOPs and HGINet at 400.0M parameters and 530.4G FLOPs.
• Robustness across attributes (Table 3): HSC-SAM records the best MAE/Adp-F in every challenge category listed: Cluttered Backgrounds (0.0033 / 0.7588), Complex Shapes (0.0026 / 0.7727), Dynamic Lighting (0.0015 / 0.8242), the column labeled "Multiple Objects" (0.0011 / 0.5828), and Occlusion (0.0012 / 0.8698).
• Hyperspectral baselines trail: Hyperspectral salient object detection methods perform far worse on this benchmark, for example SAD with MAE 0.1505 and Adp-F 0.0061, and DMSSN with MAE 0.0295.
• Ablation of components (Table 4): Starting from a Light-SAM baseline with pseudo-RGB input only (Adp-F 0.481, E 0.782), adding SSDM raises Adp-F to 0.557 (a reported 15.6% gain), adding SSCP to 0.596 (7.0% Adp-F and 2.1% E gain), adding SGTD to 0.637 (6.9% Adp-F, 1.1% E), and applying FDE only during training yields the final 0.681 Adp-F and 0.853 E (described as a 6.9% Adp-F legacy).
• SGTD threshold sensitivity (Table 5): τ = 0.01 gives the best overall balance, with MAE 0.0017, Adp-F 0.680, E 0.853, and S 0.802. Other tested values (0.1, 0.03, 0.003, 0.001) give MAE between 0.0018 and 0.0020 and Adp-F between 0.659 and 0.680.
• SGTD efficiency table (Table 6): The table lists 12.09M parameters, 19.1 FPS, and Adp-F 0.639 with SGTD, versus 11.74M parameters, 20.3 FPS (a stated 6.3% increase), and Adp-F 0.681 without SGTD. The surrounding text describes the trade-off as a substantial accuracy gain despite slightly higher parameters and lower FPS, which is the opposite direction from the Adp-F values printed in that table.
• Generalization claim: The abstract states HSC-SAM generalizes robustly to other public HSI datasets, but no specific numbers for those datasets are reported in the provided content.

Methodology in Plain English

The task is to take a hyperspectral cube (height × width × spectral channels) and output a single-channel map marking the camouflaged object.

HSC-SAM splits the hyperspectral input into two complementary views, because SAM's pretrained encoder expects ordinary three-channel images and cannot ingest 200 bands directly:

• Spatial view (SSDM, color space mapping): 33 representative bands are sampled from the 200 spectral channels and converted into the CIE XYZ color space using standard color matching functions. This produces a pseudo-color image that retains material-dependent spectral nuance while looking familiar to SAM's image encoder.
• Spectral view (SSDM, spectral saliency generator): A spectral saliency generator from prior work (SSG, Liu et al. 2023) builds an N-level Gaussian pyramid over the hyperspectral image and computes spectral angular distances between feature vectors from non-adjacent pyramid levels. Intermediate saliency maps from levels c ∈ {2, 3, 4} are concatenated into a three-channel spectral prompt.
• Token dropout (SGTD): Encoded image tokens are scored by the average activation

Authors’ abstract

RGB-based camouflaged object detection struggles in real-world scenarios where color and texture cues are ambiguous. While hyperspectral image offers a powerful alternative by capturing fine-grained spectral signatures, progress in hyperspectral camouflaged object detection (HCOD) has been critically hampered by the absence of a dedicated, large-scale benchmark. To spur innovation, we introduce HyperCOD, the first challenging benchmark for HCOD. Comprising 350 high-resolution hyperspectral images, It features complex real-world scenarios with minimal objects, intricate shapes, severe occlusions, and dynamic lighting to challenge current models. The advent of foundation models like the Segment Anything Model (SAM) presents a compelling opportunity. To adapt the Segment Anything Model (SAM) for HCOD, we propose HyperSpectral Camouflage-aware SAM (HSC-SAM). HSC-SAM ingeniously reformulates the hyperspectral image by decoupling it into a spatial map fed to SAM's image encoder and a spectral saliency map that serves as an adaptive prompt. This translation effectively bridges the modality gap. Extensive experiments show that HSC-SAM sets a new state-of-the-art on HyperCOD and generalizes robustly to other public HSI datasets. The HyperCOD dataset and our HSC-SAM baseline provide a robust foundation to foster future research in this emerging area.