DuFal: Dual-Frequency-Aware Learning for High-Fidelity Extremely Sparse-view CBCT Reconstruction

AI summary

Overview

• Research area: Medical image reconstruction / computer vision — specifically Cone-Beam Computed Tomography (CBCT) reconstruction from very few X-ray projections, combining implicit neural representations with Fourier-domain learning.
• Technical level: Advanced. The paper assumes familiarity with CT reconstruction geometry, Fourier Neural Operators, implicit neural representations, and cross-attention mechanisms.
• One-sentence scope: The paper proposes DuFal, a dual-path encoder that fuses a frequency-domain encoder (built on a new High-Local Factorized Fourier Neural Operator) with a standard spatial encoder, to reconstruct high-fidelity CT volumes from extremely sparse-view CBCT data.

What This Paper Is About

CBCT gives high-resolution 3D images but requires many X-ray projections, which means radiation dose. Reducing to extremely sparse views (defined in the paper as ≤ 10 projections) makes the reconstruction problem severely ill-posed, and conventional CNN-based encoders are biased toward low-frequency information, so they lose the fine anatomical detail — edges, boundaries, small structures such as nodules — that corresponds to high-frequency components. DuFal attacks this directly by adding an explicit frequency-domain encoding pathway, in parallel with the spatial one, so that both global and local high-frequency cues are captured before the features are decoded into a CT volume.

Key Contributions

1. DuFal framework: A complete end-to-end framework for sparse-view CBCT reconstruction that processes X-ray projections through parallel spatial and frequency pathways (a "Frequency-enhanced Dual-encoding" stage), capturing structural context alongside high-frequency anatomical detail.
2. HiLocFFNO blocks: A High-Local Factorized Fourier Neural Operator block that combines a Global High-frequency Enhanced FNO (gHiF) branch with a Local High-frequency Enhanced FNO (lHiF) branch operating on spatially partitioned patches, complementing FNO's global receptive field with local detail preservation.
3. Spectral-Channel Factorization (SCF): A weight decomposition that factorizes complex FNO weight tensors into separate channel-mixing and spectral-weighting components. The paper reports 82.20% total parameter reduction versus vanilla FNO while maintaining reconstruction quality.
4. Cross-Attention Frequency Fusion (CAFF): A fusion mechanism operating directly in the frequency domain, applying cross-attention separately to real and imaginary components, to integrate spatial and spectral features from the two encoding pathways.

The paper also claims (Table 1) that DuFal is the first end-to-end method for sparse-view CBCT reconstruction that jointly models spatial features and global and local frequency information.

Main Findings

• LUNA16, 6-view: DuFal reaches 28.20 ± 0.38 dB PSNR / 85.83 ± 0.27% SSIM, versus DIF-Gaussian at 26.48 / 81.78, FreeSeed at 25.59 / 77.36, NeRP at 23.55 / 74.46, DIF-Net at 24.68 / 71.46, NAF at 18.76 / 54.16, SART at 18.94 / 49.47, and FDK at 15.36 / 31.41.
• LUNA16, 8-view: DuFal reaches 28.77 ± 0.42 dB / 85.50 ± 1.65%, versus DIF-Gaussian at 26.93 / 82.09 and FreeSeed at 26.86 / 78.92.
• LUNA16, 10-view: DuFal reaches 29.41 ± 0.18 dB / 87.67 ± 0.22%, versus DIF-Gaussian at 29.29 / 87.55 and FreeSeed at 27.23 / 79.25.
• Parameter efficiency: In the final layer example with C_l = 512, C̄_l = 1024, and modes₁ = modes₂ = 16, the factorized SCF weight retains only 0.49% of the original parameters, roughly a 99.51% reduction. The general saving ratio is given as 1/(modes₁·modes₂) + 1/C̄_l. The contributions list states an 82.20% total parameter reduction compared with vanilla FNO.
• ToothFairy dataset: The provided excerpt shows only the partial table. Reported baseline values are: 6-view — FDK 17.07 / 39.90, SART 20.04 / 64.98, NAF 20.58 / 63.52, NeRP 21.27 / 72.06; 8-view — FDK 18.42 / 43.29, SART 21.92 / 67.86, NAF 22.39 / 67.24, NeRP 24.18 / 78.83; 10-view — FDK 19.58 / 47.21, SART 22.82 / 71.53, NAF 23.84 / 72.52, NeRP 25.99 / 82.08. DuFal's ToothFairy results are not reported in the available content, and the FreeSeed row is cut off mid-entry.
• Capability positioning (Table 1): DuFal is the only listed method marked as having spatial modelling, end-to-end reconstruction, and both global and local frequency modelling. FreeSeed is marked as local-frequency and not end-to-end; FDK, SART, NAF, NeRP, DIF-Net and DIF-Gaussian are marked with no frequency modelling.
• Baselines not compared: C2RV-Net was not publicly available at submission time; the authors provide a comparison using their own implementation in Appendix A.4.

Methodology in Plain English

The paper adopts a supervised implicit neural representation design, building on the Deep Intensity Field (DIF) framework, which treats a CT volume as a continuous intensity field.

Stage one — feature encoding. Each of the K input X-ray projections (256 × 256, with known gantry angles) is passed through two parallel encoders at the same time:

• A Spatial Encoder, which is a conventional convolutional encoder: at each depth, two 3 × 3 convolutions with ReLU followed by 2 × 2 max-pooling with stride 2 (halving resolution and doubling channel width).
• A Frequency Encoder, made of L stacked HiLocFFNO blocks. Each block has two branches. The global branch (gHiF) Fourier-transforms the feature map, retains a limited number of frequency modes while selectively keeping high-frequency components rather than the low-frequency modes vanilla FNO keeps, applies a learned complex weight, and inverts the transform. The local branch (lHiF) partitions the same input into 16 × 16 non-overlapping patches and runs the same frequency operations on each patch independently, then reassembles them — preserving spatial locality that global frequency analysis can blur. The two branch outputs are summed with a linear projection of the input, passed through GeLU, and refined by a Galerkin-Attention layer.

To keep this affordable, SCF replaces the single large complex weight tensor (which scales as C̄_l × C_l × modes₁ × modes₂) with two small tensors: one for mixing channels and one for weighting spectral locations — analogous to depthwise separable convolution.

The frequency features and spatial features at each resolution are then merged by CAFF: both are convolved and Fourier-transformed, split into real and imaginary parts, and cross-attention is applied separately to each part using the frequency components as keys and values and the spatial components as queries. The attended frequency output is added residually to the spatial frequency components and transformed back to the spatial domain. A decoder with upsampling, skip connections and 3 × 3 Conv-ReLU blocks produces one feature map E_k per view.

Stage two — intensity field decoding. This is unchanged from the DIF design. Any 3D query point is projected into all K views, view-specific features are sampled by bilinear interpolation, aggregated by a set function into a single vector, and mapped to an intensity by a four-layer MLP. Evaluating this over a uniform 3D grid yields the volume Ŷ.

Training samples N random 3D points per volume and minimizes mean squared error between predicted and ground-truth intensities (obtained by trilinear interpolation from the CT volume).

Implementation details reported: L = 4 iterative blocks, modes₁ = modes₂ = 16 global Fourier components, local branch on 16 × 16 × 16 non-overlapping patches, 4 attention heads in the fusion block, Adam optimizer with learning rate 2 × 10⁻⁴, batch size 4, cosine annealing scheduler. Datasets: LUNA16 (888 chest CT scans — 738/50/100 train/val/test — resampled to 256³ with [1.6, 1.6, 1.6] mm spacing) and ToothFairy (443 dental CBCT scans — 343/25/75 — resampled to 256³ with [0.54, 0.54, 0.21] mm spacing). Projections were simulated with the open-source TIGRE toolbox, with viewing angles uniformly sampled over 180°, at 6, 8, and 10 views. DIF-Net and DIF-Gaussian were reproduced with 400 epochs on LUNA16 and 600 epochs on ToothFairy. Evaluation uses PSNR (dB) and SSIM (%), plus weighted W-PSNR/W-SSIM variants in the ablation study.

Why This Matters

The work reframes where frequency handling belongs: rather than a post-processing denoiser hard-coded into convolutional layers (as in FreeSeed) or a late-stage attention add-on, frequency reasoning is built into the encoder itself. If the reported gains hold, extremely sparse-view CBCT — the regime with the lowest radiation dose — becomes viable without losing the fine detail clinicians need, and the SCF factorization makes that affordable in parameter count.

Real-world applications:

• Low-dose CBCT scanning protocols where radiation exposure must be minimized while retaining diagnostic image quality.
• Dental CBCT, the domain of the ToothFairy dataset used here.
• Chest CT screening and lung nodule assessment, the domain of LUNA16, where small high-frequency structures matter most.
• Image-guided and intraoperative 3D imaging, where only a few projections are physically obtainable from limited gantry angles.

Industry relevance: Medical imaging device manufacturers and clinical imaging software vendors could integrate the Frequency Encoder and CAFF as modular units, since the paper states both are designed to be plugged into existing frameworks. The parameter reduction matters for deployment on resource-constrained hardware, which the authors explicitly cite as a motivation for SCF.

Future Directions

• Complete and verify the ToothFairy results: The provided content cuts off mid-table, leaving DuFal's dental CBCT numbers and the full FreeSeed row unreported here; the full comparison is needed to confirm the cross-dataset claim.
• Cross-dataset generalization: Both evaluations use datasets with already-standardized 256³ training volumes. Whether the frequency encoder transfers to other anatomies, scanners, or native resolutions that differ from the resampled grids is an open question.
• Extending below the tested view counts: The experiments cover 6, 8, and 10 views. Performance in the 1–5 view regime, where self-supervised INR methods like NAF and NeRP are described as lacking sufficient prior data, was not tested.
• Comparison against newly available methods: C2RV-Net could not be compared with official code at submission time, and the paper's own reimplementation is relegated to an appendix; an independent head-to-head would strengthen the state-of-the-art claim.
• Cost and latency trade-offs: The paper notes that naively applying FNO significantly reduces inference speed and that SCF was introduced to address this, but no runtime or throughput figures appear in the available content.

Target Audience

Researchers and graduate students in medical image reconstruction, computational imaging, and computer vision working on sparse-view or low-dose CT, as well as practitioners implementing implicit neural representations and neural operators for 3D medical data. Readers without a background in Fourier analysis, CT projection geometry, or attention mechanisms will find the method sections dense; the paper is most valuable to those already familiar with DIF-style end-to-end reconstruction pipelines and looking for a principled way to inject high-frequency modeling into the encoder.

Authors’ abstract

Sparse-view Cone-Beam Computed Tomography reconstruction from limited X-ray projections remains a challenging problem in medical imaging due to the inherent undersampling of fine-grained anatomical details, which correspond to high-frequency components. Conventional CNN-based methods often struggle to recover these fine structures, as they are typically biased toward learning low-frequency information. To address this challenge, this paper presents DuFal (Dual-Frequency-Aware Learning), a novel framework that integrates frequency-domain and spatial-domain processing via a dual-path architecture. The core innovation lies in our High-Local Factorized Fourier Neural Operator, which comprises two complementary branches: a Global High-Frequency Enhanced Fourier Neural Operator that captures global frequency patterns and a Local High-Frequency Enhanced Fourier Neural Operator that processes spatially partitioned patches to preserve spatial locality that might be lost in global frequency analysis. To improve efficiency, we design a Spectral-Channel Factorization scheme that reduces the Fourier Neural Operator parameter count. We also design a Cross-Attention Frequency Fusion module to integrate spatial and frequency features effectively. The fused features are then decoded through a Feature Decoder to produce projection representations, which are subsequently processed through an Intensity Field Decoding pipeline to reconstruct a final Computed Tomography volume. Experimental results on the LUNA16 and ToothFairy datasets demonstrate that DuFal significantly outperforms existing state-of-the-art methods in preserving high-frequency anatomical features, particularly under extremely sparse-view settings.