Partial Ring Scan: Revisiting Scan Order in Vision State Space Models

AI summary

Overview

Research area: Computer vision architectures — specifically Vision State Space Models (Vision SSMs / Mamba-style backbones), with a focus on image serialization order ("scan order") for classification, object detection, and instance segmentation.

Technical level: Advanced. The paper assumes familiarity with selective state space models, recurrent sequence dynamics, and standard vision benchmarks, though its central idea (how you traverse an image) is conceptually simple.

Scope in one sentence: The paper argues that the traversal order used to flatten a 2D image into a 1D token sequence is a first-class design choice in Vision SSMs, and introduces a concentric-ring traversal plus channel filtering that improves ImageNet-1K accuracy, MS COCO detection/segmentation accuracy, throughput, and rotation robustness relative to fixed-path baselines.

What This Paper Is About

Vision SSMs process images by flattening a 2D grid into a 1D token sequence along a chosen path — raster, serpentine, diagonal, and so on. The authors show that this choice is not a neutral implementation detail: it determines which pixels become neighbors in the sequence, and a poorly aligned path fractures object continuity and degrades sharply when an image is rotated. The paper's goal is to design a traversal that stays geometrically stable under rotation without any rotation-specific training, and that does so while remaining linear-time and efficient.

Key Contributions

1. A systematic analysis of scan order in Vision SSMs. The authors characterize how raster and serpentine traversals (as used in VMamba, Zigma, MaIR, LocalMamba) define sequence adjacency, fracture object continuity when objects run misaligned with the path, and degrade under geometric transformations such as in-plane rotation. Twelve primitive scan orders (S1–S12, including left-to-right raster, serpentine, and diagonal) are defined for comparison in Figure 4, along with pairs of scans (S13–S18) and aggregations of four scans (S19–S21).

2. Ring Scan for Vision SSMs. A rotation-stable traversal that partitions pixels into concentric rings by Euclidean distance to the image center, performs order-agnostic aggregation within each ring (alternating clockwise for odd rings and counterclockwise for even rings), then propagates context from inner to outer rings through a short radial selective SSM.

3. Partial Channel Filtering (PCF). A hard, mean-threshold channel routing scheme: channel salience is computed by global average pooling, channels with salience at or above the mean are sent through the recurrent ring pathway, and the remainder bypass through a lightweight residual branch. This costs O(D_c) rather than the O(D_c log D_c) of a median variant.

4. A unified architecture (PRISMamba) with empirical validation. PRISMamba reports state-of-the-art results among the Vision SSM families compared, on ImageNet-1K, MS COCO detection and instance segmentation, plus stress tests for rotation and random-mask occlusion.

Main Findings

• ImageNet-1K accuracy and efficiency. At 224×224, PRISMamba reaches 84.5% Top-1 with 3.9G FLOPs, 22M parameters, and 3,054 img/s on an Nvidia A100, compared with VMamba at 82.6%, 5.6G FLOPs, 30M parameters, and 1,686 img/s. The paper states this is a 1.9-point improvement with roughly 30% fewer FLOPs and about 1.8× higher throughput.

• Channel filtering improves accuracy and speed simultaneously. The variant without PCF, PRISMamba (w/o PCF), reaches 84.1% Top-1 at 4.6G FLOPs, 27M parameters, and 2,177 img/s. Enabling PCF cuts FLOPs from 4.6G to 3.9G (−15% per the paper's text), raises throughput from 2,177 to 3,054 img/s (+40%), and raises Top-1 from 84.1% to 84.5% (+0.4 percentage points).

• PCF generalizes across Vision-Mamba families. Inserting PCF into Vim, VMamba, QuadMamba, Adventurer, SparX-Mamba, and GroupMamba lowered parameters by 2–6M, reduced FLOPs by 0.2–0.6G (roughly 4–12%), increased throughput by about 35–50%, and improved Top-1 by +0.2 to +0.4 pp. Examples: Vim 80.5% → 80.8% (5.3G → 4.9G, 811 → 1,183 img/s); GroupMamba 83.9% → 84.2% (7.0G → 6.4G, 803 → 1,075 img/s); SparX-Mamba 83.5% → 83.9% (5.2G → 4.8G, 1,370 → 1,959 img/s).

• Hard channel filtering beats soft channel attention. Relative to PRISMamba (w/o PCF) at 27M / 4.6G / 2,177 img/s / 84.1%: adding SE keeps FLOPs flat but slows inference to 2,089 img/s for +0.1 pp; adding CBAM raises parameters to 30M and slows throughput to 1,982 img/s for +0.2 pp with no FLOP reduction. PCF instead reaches 22M / 3.9G / 3,054 img/s / 84.5%.

• Rotation robustness. Under in-plane rotations of 0°/30°/60°, fixed-path baselines drop by roughly 1–2 points — VMamba falls from 82.6 to 80.7 at 30° and 80.6 at 60°; GroupMamba from 83.9 to 82.1 and 82.0; PlainMamba from 81.6 to 79.6 and 79.8; Vim from 80.5 to 78.6 at both angles. PRISMamba (w/o PCF) holds 84.1/83.9/83.9, and PRISMamba reaches 84.5 at 0° and 84.3/84.4 at 30°/60°, surpassing all compared variants at every angle.

• MS COCO detection and instance segmentation. With Mask R-CNN on MS COCO mini-val, 1× schedule, 1280×800, PRISMamba achieves 48.9 AP_box and 43.2 AP_mask at 235G FLOPs, versus VMamba at 46.5/42.1 (262G), GroupMamba at 47.6/42.9 (279G), and DefMamba at 47.5/42.8 (268G). Under the stricter AP75_box metric it reports 52.6 versus 52.1 (GroupMamba) and 51.7 (DefMamba). PlainMamba-Adapter uses 542G FLOPs for 46.0 AP_box.

• Robustness to random-mask occlusion. With a 16×16 masked tile, VMamba drops from 82.6% to 81.7% (30M) and from 83.6% to 82.3% (50M); LocalMamba-T drops from 82.7% to 81.9% and 83.7% to 82.5%. PRISMamba degrades by ≤0.6 points, going from 84.5% to 84.1% (30M) and 85.3% to 84.7% (50M); with a 4×4 mask the drops are 0.1 points in both cases.

• Design rationale reported for the rotation result. In-plane rotations leave each pixel's ring index unchanged, so a ring's loop only experiences a cyclic shift; alternating traversal direction across rings reduces start-index bias. The authors state this yields rotation robustness without polar remapping or rotation-specific augmentation.

Methodology in Plain English

The core observation is that an SSM reads pixels one at a time and carries a hidden state forward, so the path determines which pixels influence which others. A row-by-row path makes horizontal neighbors adjacent but is broken by rotation, which adds padding and globally reindexes the image.

The authors replace the path with a distance-based grouping. Each pixel is assigned a ring index by dividing its Euclidean distance from the image center by a ring width Δr. Within a ring, a fixed closed loop visits the pixels and a small selective SSM runs around that loop; outputs are averaged into a single ring descriptor, so the ordering inside the ring does not matter. Rings alternate direction (clockwise for odd, counterclockwise for even). The resulting descriptors form a short radial sequence processed by a second selective SSM from the innermost ring outward, and the outputs are written back to the grid with a 1×1 projection and fused residually with the backbone stream.

Because rings have far fewer members than the number of pixels, and the radial chain is short, the added recurrence cost stays small relative to scanning the entire grid.

On top of this, Partial Channel Filtering computes a global-average-pooled salience per channel and compares it to the mean salience. Channels above the threshold go through the ring modules; the rest take an identity/residual path and are fused later. The center can optionally be supplied by an object detector, making the descriptor object-aware.

For evaluation, models were trained from scratch with AdamW for 300 epochs, batch size 128, a five-epoch linear warm-up, momentum 0.9, cosine learning rate schedule with initial learning rate 1×10⁻⁴, and the same augmentations as prior work (mixup, random erasing, auto-augmentation). Throughput was measured on an Nvidia A100 GPU. Comparisons cover fifteen Vision-Mamba families: Vim, VMamba, SiMBA, Zigma, QuadMamba, LocalMamba, FractalMamba, Adventurer, SparX-Mamba, EfficientVMamba, PlainMamba, GroupMamba, VSSD, DefMamba, and MaIR.

Why This Matters

Impact on research. The paper reframes scan order from an implementation detail into a design variable that mediates the alignment between sequence adjacency (what the SSM processes) and geometric adjacency (the true image structure). It also shows that channel allocation and traversal design are coupled levers rather than independent choices, since the same filter that reduces FLOPs also improves accuracy. The generality result — PCF improving six other backbones — suggests a cheap plug-in rather than an architecture-specific trick. Code is stated to be released upon acceptance, and the paper notes that ablation studies on other scanning orders in Figure 4 will appear in the Supplementary Material rather than in this content.

Real-world applications:

• Aerial and satellite imagery, where scene orientation is arbitrary and the related work notes that complex multi-directional scans do not consistently beat simple rasterization.
• Medical imaging, where rotation invariance matters for whole-slide and volumetric data and where VM-UNet-style designs are already used for segmentation.
• Robotics and autonomous perception, where camera roll and mounting angle change apparent orientation at test time.
• Edge and mobile deployment, where the reported FLOPs and throughput improvements matter for latency budgets; hardware-aware selective scanning for mobile and edge is already an active line in the related work.

Industry relevance. The reported gains come without extra training data, without rotation-specific augmentation, and without increasing model size — a favorable profile for teams that need to improve an existing Vision SSM backbone under fixed compute. The throughput numbers are measured on an A100 GPU and are hardware-specific, and PCF is described as requiring no architectural surgery or re-training tricks.

Future Directions

1. Learning ring origins and widths. The current implementation uses a fixed image center and a discrete ring width, which the authors note may be suboptimal for off-center subjects or extreme aspect ratios. Learning these, or using content-adaptive ring partitions, is proposed.
2. Coupling ring traversal with inpainting priors or anti-aliasing warps to improve behavior under extreme rotations, where large padded regions still reduce valid information.
3. Extending rings to spatio-temporal and volumetric settings — video and 3D medical volumes are named — and to object-aware traversals where a detector supplies the center.
4. Open question: how much of the gain comes from the ring geometry versus the channel filter. The paper isolates these with the with/without-PCF comparison and with SE/CBAM, but the ring design itself is only ablated against other scan orders in results deferred to the Supplementary Material.

Target Audience

Researchers and engineers working on Vision Mamba and state space models for vision, who will find the scan-order analysis and the PCF plug-in directly actionable. It is also relevant to practitioners who care about the accuracy–throughput trade-off on classification or COCO-style dense prediction, and to anyone studying robustness to rotation or occlusion. Readers need some background in selective SSMs and sequence serialization to follow the method section, but the core insight — that traversal order shapes what the model can see locally — is accessible without it.

Authors’ abstract

State Space Models (SSMs) have emerged as efficient alternatives to attention for vision tasks, offering lineartime sequence processing with competitive accuracy. Vision SSMs, however, require serializing 2D images into 1D token sequences along a predefined scan order, a factor often overlooked. We show that scan order critically affects performance by altering spatial adjacency, fracturing object continuity, and amplifying degradation under geometric transformations such as rotation. We present Partial RIng Scan Mamba (PRISMamba), a rotation-robust traversal that partitions an image into concentric rings, performs order-agnostic aggregation within each ring, and propagates context across rings through a set of short radial SSMs. Efficiency is further improved via partial channel filtering, which routes only the most informative channels through the recurrent ring pathway while keeping the rest on a lightweight residual branch. On ImageNet-1K, PRISMamba achieves 84.5% Top-1 with 3.9G FLOPs and 3,054 img/s on A100, outperforming VMamba in both accuracy and throughput while requiring fewer FLOPs. It also maintains performance under rotation, whereas fixed-path scans drop by 1~2%. These results highlight scan-order design, together with channel filtering, as a crucial, underexplored factor for accuracy, efficiency, and rotation robustness in Vision SSMs. Code will be released upon acceptance.