Uncertainty-Guided Cross-Modal Alignment for Diffusion-Based IoT Intrusion Detection

AI summary

Overview

Research area: Machine learning for network security, specifically generative data augmentation for imbalanced Internet-of-Things (IoT) intrusion detection using conditional diffusion models and multimodal latent alignment.

Technical level: Advanced. The paper assumes familiarity with variational autoencoders (VAEs), latent diffusion models, the EDM (Karras et al., 2022) training/sampling framework, and standard imbalance baselines such as SMOTE.

Scope (one sentence): The paper proposes a reliability-weighted cross-modal alignment rule — where each modality's normalized VAE posterior variance controls how strongly it acts as a teacher for the other encoders — and evaluates it as the geometry-shaping step before class-conditional diffusion generates synthetic minority-class (attack) traffic on TON-IoT.

What This Paper Is About

Diffusion models generate data by pushing noise toward a learned distribution, so their behavior depends heavily on the geometry of the latent space in which denoising happens. When several independently trained encoders (here: tabular features, a DeepInsight image, and a packet sequence) each map the same network event into their own latent region, those regions may not line up — and naive uniform alignment lets a noisy or ambiguous view drag the shared representation off target. The paper's goal is to make alignment reliability-aware: each view's own reported uncertainty decides how much cross-modal influence it is allowed to exert, and the resulting joint latent space is then used to generate synthetic attack samples that improve detection on genuinely imbalanced real data.

Key Contributions

1. A bounded, sample-specific asymmetric alignment rule. Each modality's normalized VAE posterior variance is converted into a confidence score via a softmax with temperature γ = 2, then blended with a uniform floor via η = 0.5 so no view ever receives zero weight. Weights are detached so an encoder cannot lower its own loss by manipulating its reported variance.

2. A stop-gradient teacher formulation. In the alignment loss, the target modality's weight (not the source's) controls the update — a lower-uncertainty view becomes a stronger teacher. The objective sums weighted squared distances between a modality's 32-dimensional projection and the stop-gradient projection of every other modality.

3. A three-view joint latent pipeline for IoT traffic. Tabular flow attributes, an 8×8 DeepInsight image, and a 50-packet sequence are encoded by Transformer, convolutional, and LSTM VAEs (full latent means of dimension 92, 32, and 16), standardized blockwise, concatenated into a 140-dimensional vector, and modeled by a class-conditional EDM.

4. A controlled ablation isolating the weighting rule. Uniform and uncertainty-weighted variants share encoders, latent dimensions, denoiser, optimizer settings, class condition, sampler, augmentation volume, data split, and identical alignment fine-tuning — they differ only in the target weights, so the comparison measures the weighting rule itself.

Main Findings

• Generative fidelity (Table 1, PRDC on tabular attacks, mean ± std over three seeds): Uncertainty-aware alignment has the highest mean in all four columns — precision 0.7341 ± 0.0316, recall 0.9067 ± 0.0194, density 0.7229 ± 0.0403, coverage 0.7921 ± 0.0185 — versus uniform alignment (0.7248 ± 0.0137, 0.9054 ± 0.0181, 0.7090 ± 0.0154, 0.7696 ± 0.0063) and no alignment (0.6688 ± 0.1015, 0.8563 ± 0.0775, 0.6616 ± 0.0969, 0.7093 ± 0.0897).

• Uniform alignment stabilizes fidelity: Uniform alignment improves every PRDC mean over independent latents and cuts coverage standard deviation from 0.0897 to 0.0063.

• Higher variance on three metrics: The uncertainty-aware variant's standard deviations exceed the uniform variant's for three of the four PRDC metrics, so the authors characterize the result as better mean fidelity rather than uniformly lower variance.

• Downstream detection (Table 2, untouched real, imbalanced test set): Uncertainty-aware alignment reaches ROC-AUC 0.9515 ± 0.0003, PR-AUC 0.8954 ± 0.0006, balanced accuracy 0.8395 ± 0.0024, and attack-class F1 0.7940 ± 0.0022 — best mean in every column.

• Gains over the matched variant: Against uniform alignment it adds 0.0129 ROC-AUC, 0.0204 PR-AUC, 0.0150 balanced accuracy, and 0.0143 F1; against the strongest alternative in each column the gains are 0.0097, 0.0155, 0.0085, and 0.0119 respectively.

• Class balancing alone is insufficient: Class weighting (BAcc 0.8212, F1 0.7213) and random oversampling (BAcc 0.8207, F1 0.7209) reduce balanced accuracy and F1 relative to real-only training (BAcc 0.8244, F1 0.7821); SMOTE improves balanced accuracy (0.8310) but not the other three measures.

• Alignment alone is not enough: No-alignment diffusion (ROC-AUC 0.9381, PR-AUC 0.8747, BAcc 0.8243, F1 0.7800) and uniform-alignment diffusion (0.9386, 0.8750, 0.8245, 0.7797) are nearly indistinguishable downstream — uniform alignment improves fidelity but not the classifier.

• Stated caveat: With only three seeds the authors treat the differences as controlled evidence, not statistical significance, and describe the variance signal as a heuristic encoder-derived reliability proxy rather than a calibrated probabilistic uncertainty.

Methodology in Plain English

The researchers start with the network-traffic portion of TON-IoT and build a fixed 30,000-event pool (21,000 training / 3,000 validation / 6,000 test, with 16,800 normal and 4,200 attack events in training). Each event is described three ways: 22 flow features (also rearranged into an 8×8 DeepInsight image), and a 50-packet sequence with eight attributes per packet — with shared indices keeping the three views tied to the same event.

Each view gets its own autoencoder that compresses it into a compact latent code. Crucially, each autoencoder also reports how uncertain it is about that code (its posterior variance). The team turns that uncertainty into a weight per sample per modality: more uncertain views get less say, but never zero say, and the weights are frozen so an encoder cannot game them. During a short alignment phase, small projection heads pull the three modalities' codes toward each other in a shared 32-dimensional space, with the more confident view acting as the fixed teacher and the less confident one doing the moving. Extra penalties keep the reconstructed inputs accurate and stop the shared space from collapsing.

After alignment, the full latent codes are rescaled with training statistics and glued into one 140-dimensional vector, which a class-conditional diffusion model learns to generate for the attack class only. At generation time the model starts from noise and takes 50 second-order sampling steps toward an attack-conditioned latent, which is split back apart and decoded into three synchronized views; a candidate is kept only if its decoded label matches the requested attack condition. Synthetic attacks are added only to the training split to equalize class counts, and a single fixed XGBoost configuration is trained and scored on the untouched, still-imbalanced real test set. Everything fitted — preprocessing, image coordinates, normalization, latent statistics, uncertainty normalization, and checkpoint selection — uses training data only. All comparisons use three model seeds over a fixed saved split.

Why This Matters

Impact on research. The paper separates two things that are often conflated in multimodal generative modeling: making representations compatible and making them trustworthy. Its controlled ablation shows uniform alignment can improve generative fidelity scores while producing essentially no downstream benefit, which suggests fidelity benchmarks alone are a weak proxy for whether synthetic data helps a detector. It also carries reliability weighting — previously used in multimodal fusion — into the geometry on which conditional denoising is learned, rather than into a fusion head.

Real-world applications (bullets):

• Rare-attack detection in deployed IoT networks, where attack traffic is scarce relative to normal traffic and collecting more real attacks is impractical or unsafe.
• Security operations centers that need classifiers trained on balanced data without discarding or duplicating the normal-traffic majority class.
• Testbed and simulation augmentation for industrial or smart-city IoT deployments, where synchronized multi-sensor views (flow records, packet captures, visual summaries) describe the same event.
• Model retraining under distribution shift, where synthetic minority samples can be regenerated on demand rather than archived.

Industry relevance. The pipeline is designed around operational constraints: augmentation is confined to the training split, the test set stays real and imbalanced, the downstream detector is a standard XGBoost model rather than a bespoke architecture, and all fitting decisions avoid test data. The reported gains over class weighting, random oversampling, and SMOTE — the default tools in most production imbalance workflows — are the practically relevant comparison.

Future Directions

• Extend beyond binary detection: the authors plan to evaluate multiclass and cross-dataset settings; the present study collapses all attack categories into a single binary target on TON-IoT only.
• Analyze the full denoising trajectory: the current work reports endpoint measurements and motivates future study of path length, curvature, solver error, and cross-modal agreement throughout sampling.
• Joint multimodal fidelity: PRDC is reported only for the tabular attacks, so a metric covering all three generated views together remains open.
• More formal uncertainty: replacing the heuristic posterior-variance proxy with calibrated probabilistic uncertainty measures, and assessing denoising stability, is explicitly listed as future work.

Target Audience

Researchers and graduate students working on generative augmentation, multimodal representation learning, or latent diffusion — particularly those applying these methods to security, fraud, or other severely imbalanced domains. It also suits applied ML engineers in network security who already know SMOTE and class weighting and want to understand when synthetic minority generation actually beats them, and practitioners interested in how uncertainty signals from autoencoders can be used as a curriculum for aligning imperfect, heterogeneous data sources. Readers without background in VAEs and diffusion sampling will find the method sections dense.

Authors’ abstract

Diffusion models evolve noisy states toward a learned data distribution, so their sampling behavior depends on the geometry of the representation in which denoising occurs. This dependence is especially important for multimodal data: independently trained encoders can map synchronized views to incompatible latent regions, while uniform alignment can let an uncertain view distort the shared geometry. To address this gap, we introduce a novel reliability-guided alignment rule that uses normalized VAE posterior variance to control directional cross-modal influence before diffusion training. We study this issue in minority-attack generation for imbalanced Internet-of-Things intrusion detection. Tabular flow attributes, a DeepInsight image, and a packet sequence are encoded by modality-specific variational autoencoders. Per-modality normalized VAE posterior variances provide bounded, sample-specific weighting signals in an asymmetric stop-gradient alignment objective. We use these quantities as heuristic encoder-derived reliability proxies rather than calibrated probabilistic uncertainties. The full modality-specific latent means obtained after alignment fine-tuning are standardized and modeled jointly by a class-conditional EDM; a second-order sampler then generates paired attack-conditioned latent blocks for the three decoders. In controlled experiments on TON-IoT, uncertainty-aware alignment obtains the highest mean precision, recall, density, and coverage among three diffusion variants. Augmenting only the training split yields ROC-AUC 0.9515, PR-AUC 0.8954, balanced accuracy 0.8395, and attack-class F1 0.7940 on an untouched real test set. Among the two variants with matched alignment fine-tuning, posterior-variance-based weighting improves both tabular generative fidelity and downstream detection relative to uniform weighting.