Mixed Data Clustering Survey and Challenges

AI summary

Overview

• Research area: Machine learning, specifically clustering of mixed (numerical + categorical) data, with a secondary focus on dimensionality reduction and clustering evaluation.
• Technical level: Intermediate. The survey portions are accessible, but the algorithm sections use substantial formalism (mixture models, latent variable models, graph Laplacians, kernel density estimation).
• Scope: A survey of dimensionality reduction, mixed-data clustering, and clustering evaluation methods, combined with the presentation of a new pretopology-based hierarchical clustering algorithm and a benchmarking effort.

Note on completeness: the paper content provided is truncated inside Section 4.4. Sections covering the remaining algorithms (including the pretopology algorithm in 4.8), the evaluation measures (Section 5), the datasets (Section 6), the experimental results (Section 7), the discussion of challenges (Section 8) and the conclusion (Section 9) are not included. Benchmark numbers, dataset sizes and comparative results therefore cannot be reported here, and this summary flags where that information would have appeared.

What This Paper Is About

The big data paradigm brings unprecedented volume, velocity and variety, and with that variety comes mixed datasets containing both numerical and categorical variables. Traditional clustering methods were built for homogeneous numerical data and often rely on Euclidean space, so they do not directly apply to mixed data. The paper surveys the methods that have been adapted to close this gap and proposes a new pretopology-based clustering approach that builds a hierarchical, explainable clustering of mixed datasets without requiring dimensionality reduction.

Key Contributions

The authors enumerate six contributions; they are summarized here.

1. Three state-of-the-art reviews: a review of dimensionality reduction methods, a review of mixed-data clustering methods, and a review of clustering evaluation measures.
2. A new pretopology-based clustering algorithm, which the authors present as a method that bridges the gap between traditional methodologies and the complexities of mixed data.
3. A benchmarking effort comparing the surveyed methods against traditional numerical clustering methods and pretopological approaches, using datasets and a synthetic data generator developed for the purpose.
4. An in-depth discussion of remaining challenges in mixed-data clustering in the era of big data.

Main Findings

• Mixed data is not handled natively by most algorithms. Because some algorithms cannot process mixed data directly, the paper introduces dimensionality reduction methods as a preprocessing step. Several of the reviewed techniques (Laplacian Eigenmaps, UMAP) are adapted using Huang's distance, defined as d_ij = d_ij^N + γ d_ij^C, where the numerical term is a squared Euclidean distance over numerical features and the categorical term is the Hamming distance over categorical features, with γ proportional to the average standard deviation of numerical features.

• Four clustering paradigms structure the field. Following the survey by Ahmad et al., the paper distinguishes partitional, hierarchical, model-based and neural-network-based clustering. Neural-network-based methods typically use a loss combining a network loss (reconstruction or information preservation) and a clustering loss (encouraging well-separated clusters in the latent space).

• Dimensionality reduction methods trade interpretability against structure preservation. FAMD computes explained inertia directly and requires no hyperparameter tuning, avoiding potential instability, but it can be limited when there are too few observations (unstable MCA) or when numerical features are far outnumbered by categorical ones. Laplacian Eigenmaps requires hyperparameters such as t that substantially affect the outcome, and its axes lack explicit interpretation. UMAP preserves local structure and can produce well-defined clusters, but the number of neighbors k significantly impacts results and may bias interpretation. PaCMAP preserves both local and global structure, but initialization affects its results and preserving global structure may provide no additional benefit in some cases.

• Partitional clustering has known weaknesses on mixed data. K-prototypes, described as the most widely used partitional algorithm for mixed datasets, combines K-Means numerical dissimilarity with K-Modes categorical dissimilarity, but its Hamming-based categorical distance treats all mismatches equally and prototype updates can converge to local optima sensitive to initialization. Convex K-Means (Modha–Spangler) automatically identifies feature weights through exhaustive grid-search on the interval [0,1], which the authors identify as its main drawback; they state it "does not meet our needs and often fails to provide satisfactory results on large datasets."

• Model-based methods avoid user-specified feature weights. KAMILA combines k-means with a Gaussian–multinomial mixture model and balances numerical and categorical effects without requiring the user to specify relative weights; using kernel density estimation relaxes the Gaussian assumption for numerical data. ClustMD uses a latent variable model that unifies numerical, ordinal and categorical variables into a single latent representation. MixtComp is also implemented and reviewed.

• Hierarchical clustering is positioned as the interpretability answer. The paper argues hierarchical clustering aligns well with mixed-data complexity by revealing nested patterns, and that its tree-like structure lets stakeholders understand the logic behind clustering decisions — described as crucial in sensitive domains.

• Pretopology offers four stated advantages for mixed data: no dimensionality reduction required, customizable logical space creation, hyperparameters for clustering and division conditions allowing tailored hierarchy creation, and explainable AI on the dendrogram by comparing cluster characteristics grouped at each branching.

• A concrete illustration is reported for FAMD. Figure 1(a) shows FAMD in 2D on the Palmer Penguins dataset with an explained inertia of 61.40%. Figure 1(b) shows Laplacian Eigenmaps 2D with t = 1, Figure 1(c) shows UMAP 2D with k = 15, and Figure 1(d) shows PaCMAP in 2 dimensions on the Palmer Penguins dataset with FAMD initialization.

• Dataset and benchmark results are not reported in the supplied content. Section 6 (datasets) and Section 7 (results) are not included, so no dataset sizes, benchmark scores, or comparative findings between the proposed pretopology method and the other algorithms can be verified or reproduced here.

Methodology in Plain English

The researchers took a survey-plus-proposal approach.

They began by explaining why big data breaks conventional clustering: traditional methods assume homogeneous numerical data and a Euclidean space, while real datasets mix numbers with categories, sometimes also temporal variables. Treating the two kinds of variables separately ignores their interdependencies.

They then organized the field into four clustering paradigms and catalogued the specific algorithms they would implement. Every algorithm was implemented with a consistent input–output interface in Python or taken from frequently updated GitHub repositories, so the algorithms could be plugged into the same pipeline. The algorithms fall into three groups: dimensionality reduction (FAMD, Laplacian Eigenmaps, UMAP, PaCMAP), partitional clustering (K-prototypes, Convex K-Means), model-based clustering (KAMILA, ClustMD, MixtComp) and hierarchical clustering (Philip and Ottaway, DenseClus, pretopology). Source code is available at the GitHub repository https://github.com/ClementCornet/Clustering-Mixed-Data/.

To make algorithms that expect numerical input work on mixed data, the authors use Huang's distance to combine a squared Euclidean component over numerical features with a Hamming distance component over categorical features, weighted by a γ parameter tied to the average standard deviation of the numerical features. This adaptation is used in both Laplacian Eigenmaps and UMAP.

Their own proposed method uses pretopology, described as an emerging field combining topological principles with data analysis. Instead of projecting the data into a lower-dimensional numerical space, pretopology constructs a logical space that accounts for the inherent relationships between different data types, allowing numerical, categorical and temporal variables to sit within a single unified framework. The result is a hierarchical clustering — a dendrogram — whose branching points can be examined to see which cluster characteristics were grouped together, giving an explainable structure. The authors also note they use dimensionality reduction for evaluation and visualization of mixed-data clusters precisely because traditional cluster-quality evaluation techniques designed for numerical data are hard to apply here.

Why This Matters

Research impact. The paper consolidates a fragmented literature — dimensionality reduction, mixed-data clustering, and evaluation measures — into one reference with consistent terminology and notation, and releases a uniform implementation interface. That makes comparisons between methods reproducible rather than anecdotal. Its most distinctive move is importing pretopology into mixed-data clustering as an alternative to the dimensionality-reduction-then-cluster pipeline that dominates the field.

Real-world applications (as identified or implied by the paper):

• Healthcare: named explicitly by the authors as a sensitive domain where transparency and accountability are paramount, and where mixed patient data (numerical measurements plus categorical attributes) is routine.
• Finance: also named explicitly as a sensitive domain where explainable clustering supports trust and confidence in decisions.
• Decision-making across industries: the paper stresses informed decision-making and interpretable results generally in the big data era.
• High-volume industrial data: the authors' affiliation with Energisme, an energy data company, and the paper's emphasis on scalability suggest energy and operational data as a practical setting.

Industry relevance. The stated advantages of the proposed approach — no dimensionality reduction, customizable logical space, and explainability directly on the dendrogram — target exactly what regulated and high-stakes sectors need: clustering results a stakeholder can inspect and justify rather than a black-box embedding. The paper's criticism of Convex K-Means failing on large datasets signals that scalability, not just accuracy, is a gating criterion for adoption.

Future Directions

1. Handling temporal variables. The authors claim pretopology allows numerical, categorical and temporal variables in one framework, but the detail of how temporal variables are treated is in the truncated sections and is not verifiable from the supplied content.
2. Cluster quality evaluation for mixed data. The paper states that judging the quality and significance of mixed-data clusters is difficult because traditional numerical techniques do not apply. A principled evaluation scheme for mixed clusters remains a stated open problem.
3. Improving categorical distance. The identified weakness of Hamming-style distances treating every mismatch as equal, and the sensitivity of prototype-based methods to initialization and local optima, remain unresolved directions.
4. Scaling to genuinely large data. Since Convex K-Means is dismissed for its brute-force weight search and poor large-dataset results, and since UMAP's k and Laplacian Eigenmaps' t strongly affect outcomes, hyperparameter robustness and computational cost at big-data scale are open questions.

The paper's dedicated discussion of remaining bottlenecks is Section 8, which is not included in the content provided.

Target Audience

• Machine learning researchers working on clustering, heterogeneous data, or interpretable unsupervised learning, who need a consolidated map of the mixed-data landscape and its evaluation gaps.
• Data scientists and engineers in industry who must cluster real mixed datasets and want a comparison of ready-to-use implementations, including the shared GitHub codebase.
• Practitioners in regulated sectors such as healthcare and finance, where the explainability of a hierarchical dendrogram is a requirement rather than a nicety.
• Graduate students and newcomers to the field who need a structured introduction covering the four clustering paradigms, the main mixed-data distance measures, and the tradeoffs between dimensionality reduction techniques.

Readers looking specifically for benchmark results, dataset descriptions, or the full specification of the pretopology algorithm should consult the complete paper, as those sections fall outside the content available here.

Authors’ abstract

The advent of the big data paradigm has transformed how industries manage and analyze information, ushering in an era of unprecedented data volume, velocity, and variety. Within this landscape, mixed-data clustering has become a critical challenge, requiring innovative methods that can effectively exploit heterogeneous data types, including numerical and categorical variables. Traditional clustering techniques, typically designed for homogeneous datasets, often struggle to capture the additional complexity introduced by mixed data, underscoring the need for approaches specifically tailored to this setting. Hierarchical and explainable algorithms are particularly valuable in this context, as they provide structured, interpretable clustering results that support informed decision-making. This paper introduces a clustering method grounded in pretopological spaces. In addition, benchmarking against classical numerical clustering algorithms and existing pretopological approaches yields insights into the performance and effectiveness of the proposed method within the big data paradigm.