Deep Incomplete Multi-View Clustering via Hierarchical Imputation and Alignment

AI summary

Overview

Research area: Unsupervised machine learning, specifically deep incomplete multi-view clustering (IMVC) — grouping data when only some views (modalities) are observed for each sample.

Technical level: Advanced. The paper assumes familiarity with autoencoders, contrastive learning, energy-based models, and clustering evaluation metrics.

Scope: The paper introduces DIMVC-HIA, an end-to-end deep framework that combines a two-stage hierarchical imputation procedure with two alignment losses (energy-based and contrastive assignment) to cluster multi-view data under missing-view ratios from 0.1 to 0.7.

What This Paper Is About

Real-world multi-view data — the same objects described by different sensors, feature sets, or modalities — frequently has entire views missing for some samples. Existing approaches split into imputation-based methods, which can amplify their own mistakes in a self-reinforcing cycle, and imputation-free methods, which struggle as missingness grows. The authors' goal is a single framework that imputes missing information without letting errors propagate while still producing tight, semantically consistent clusters.

Key Contributions

1. DIMVC-HIA framework. An end-to-end architecture that jointly optimizes view-specific representation learning, hierarchical imputation, and dual-alignment regularization, positioned to bridge imputation-based and imputation-free paradigms.

2. Hierarchical imputation strategy. A two-stage procedure that first recovers missing cluster assignment distributions using inter-view contrastive similarity, then reconstructs missing latent features conditioned on those imputed assignments using intra-view, intra-cluster statistics.

3. Energy-based semantic alignment. Cluster-specific energy models that assign scalar energy to features, with training minimizing the deviation of each feature's energy from the lowest-energy anchor in its cluster, promoting intra-cluster compactness.

4. Contrastive assignment alignment. A loss combining contrastive alignment of cluster assignment distributions across views with an entropy regularizer on average cluster proportions, intended to improve cross-view consistency and produce confident, well-separated predictions.

Main Findings

• Highest reported accuracy on four benchmarks at low missingness: At η = 0.1, DIMVC-HIA reaches 98.40 ACC on BDGP, 99.10 on MNIST-USPS, 98.84 on Fashion-MNIST, and 96.85 on Handwritten. The authors state the model outperforms most of the ten baselines.

• Advantage grows with missingness: At η = 0.7, DIMVC-HIA achieves 95.27% ACC on Fashion-MNIST (stated as 8.53% higher than ProImp) and 94.05% on Handwritten (stated as 7.95% higher than GIMVC).

• Stability on BDGP and MNIST-USPS: The paper reports an accuracy variation of only around 6% across missing ratios on these two datasets, compared with DSIMVC's ACC on BDGP falling from 98.00% to 91.12% as η increases.

• Not a uniform win on every metric: At MNIST-USPS with η = 0.7, DIMVC-HIA's NMI of 82.16 is below DSIMVC (85.29) and ProImp (86.17), while its ACC of 93.66 and PUR of 93.66 lead that row.

• Contrastive assignment alignment is the most critical component: In the ablation study on Fashion-MNIST at η = 0.5, removing L_CAA caused the most significant performance drop; removing L_EBM or L_REC also caused considerable degradation. Numerical ablation values are not reported in the text (results appear only in Figure 2).

• Cluster structure visible under severe missingness: T-SNE embeddings showed distinct, compact clusters with well-defined boundaries on BDGP at η = 0.5 and visibly separable structure on Handwritten at η = 0.7.

• Stable convergence: On MNIST-USPS and Fashion-MNIST at η = 0.1, the loss declined rapidly within the first 25 epochs, then reduced more slowly to a stable value.

• Hyperparameter robustness: On BDGP and Handwritten at η = 0.3, ACC stayed consistently strong for α in [0.01, 0.10] and β in [0.01, 0.05]; defaults of α = 0.1 and β = 0.01 were used across all datasets.

Methodology in Plain English

The model treats each view separately at first. A view-specific autoencoder compresses each view into a latent feature vector and reconstructs it, but only observed samples contribute to the reconstruction loss. A single clustering predictor shared across all views turns each latent feature into a soft distribution over K clusters.

Imputation happens in two ordered steps rather than all at once:

1. Fill in cluster assignments first. For every pair of views, the model takes samples present in both, computes a cross-view similarity between their soft assignments, and scores how well the two views agree. This score is contrastive: a sample's own match across views counts as positive, and dissimilar others count as negative, except that samples predicted into the same cluster are excluded so they are not wrongly penalized. Views are then ranked by this score for each target view, and a missing sample's assignment is copied from the highest-ranked view in which that sample is actually available.

2. Then fill in features. Each missing sample is assigned to its most likely cluster from the imputed distribution, and its latent feature is replaced by the centroid of observed features in that view that share that cluster.

Two alignment losses then shape training. The energy-based loss learns one energy function per cluster and pushes every feature's energy toward the minimum energy observed in its cluster, so features gather around reliable anchors rather than a fixed center. The contrastive assignment loss pulls matching samples' assignment distributions together across views, weighted by the cross-view similarity score, and adds an entropy term on the average cluster distribution to prevent degenerate all-in-one-cluster solutions. The total objective sums the reconstruction loss with the two alignment terms weighted by α and β.

Settings: encoder architectures [d_v, 256, 512, d] for BDGP and Fashion-MNIST and [d_v, 256, 512, 1024, d] for MNIST-USPS and Handwritten, with d = 2000; a shared MLP of [d, 1024, K]; cluster EBMs with hidden dimensions [256, 256, 256]; batch sizes of 200 (BDGP, Handwritten), 100 (Fashion-MNIST), and 50 (MNIST-USPS); learning rate 0.0001; 100 pre-training epochs plus 200 fine-tuning epochs. Final labels come from summing the imputed assignment distributions across views and taking the argmax.

Why This Matters

Research impact. The work offers a concrete alternative to the usual trade-off between imputing missing views (risk of error propagation) and ignoring them (risk of misalignment and uncertainty). Its most distinctive move is ordering the imputation — semantics first, geometry second — so feature reconstruction is guided by recovered cluster structure rather than the other way around. The energy-based alignment is also an unusual choice for clustering regularization, shaping a continuous energy landscape instead of pulling samples toward a fixed centroid.

Real-world applications:

• Multimedia analysis, where the same content arrives as different modality streams and some streams drop out.
• Bioinformatics, where different assay types may be unavailable for different patients or samples.
• Social network mining, where user profiles are assembled from heterogeneous sources with uneven coverage.
• Sensor networks and industrial monitoring, where hardware failure, corruption, or transmission error removes entire data streams.

Industry relevance. The framework targets exactly the conditions that make deployed multi-view systems brittle — sensor failure, corrupted uploads, and preprocessing artifacts — and its reported behavior at 0.7 missing ratio is the regime where production pipelines usually break down. The code is released at https://github.com/YMBest/DIMVC-HIA, and the reported training setup (a single NVIDIA RTX 3080 GPU with 32 GB RAM) indicates the method is tractable on commodity hardware.

Future Directions

• Resolve the metric-level trade-offs. DIMVC-HIA leads ACC and PUR on MNIST-USPS at η = 0.7 but trails DSIMVC and ProImp on NMI; understanding this gap is an open question the paper does not address.
• Scale the imputation source. Assignment imputation draws from a single best-matching available view (the first available view in the similarity ranking); whether combining multiple reference views improves results is untested.
• Extend beyond the tested missingness model. All results come from randomly omitting a proportion of samples per view at η ∈ {0.1, 0.3, 0.5, 0.7}; structured, view-dependent, or block-missing patterns are not evaluated.
• Move beyond non-parametric feature imputation. Missing features are replaced by cluster prototypes, a choice the authors describe as non-parametric and stabilized through the alignment losses and reconstruction on observed samples; learned or uncertainty-aware imputation is a natural extension.

Target Audience

Graduate students and researchers working on multi-view learning, incomplete data, and deep clustering; practitioners building systems that must tolerate missing modalities, such as multimodal retrieval, sensor fusion, and healthcare data integration; and readers interested in energy-based models or contrastive objectives applied to unsupervised clustering. Comfort with autoencoders, contrastive losses, and standard clustering metrics (ACC, NMI, Purity) is assumed.

Authors’ abstract

Incomplete multi-view clustering (IMVC) aims to discover shared cluster structures from multi-view data with partial observations. The core challenges lie in accurately imputing missing views without introducing bias, while maintaining semantic consistency across views and compactness within clusters. To address these challenges, we propose DIMVC-HIA, a novel deep IMVC framework that integrates hierarchical imputation and alignment with four key components: (1) view-specific autoencoders for latent feature extraction, coupled with a view-shared clustering predictor to produce soft cluster assignments; (2) a hierarchical imputation module that first estimates missing cluster assignments based on cross-view contrastive similarity, and then reconstructs missing features using intra-view, intra-cluster statistics; (3) an energy-based semantic alignment module, which promotes intra-cluster compactness by minimizing energy variance around low-energy cluster anchors; and (4) a contrastive assignment alignment module, which enhances cross-view consistency and encourages confident, well-separated cluster predictions. Experiments on benchmarks demonstrate that our framework achieves superior performance under varying levels of missingness.