Categorical and Mixed-Type Clustering

A customer table mixes age, country, device, plan, and purchase counts. Ordinary K-Means answers with centroids carrying fractional categories, and a one-hot country field quietly outvotes two carefully engineered behavior features. The usual reflex is to flatten the table until some algorithm will accept it.

That reflex has a price. Costa and colleagues open their benchmark of mixed-type methods by naming it: “Among the most simple and intuitive strategies for clustering mixed-type data is to convert all variables to a single type, continuous or categorical, via discretization, dummy-coding or fuzzy-coding. Such a strategy may lead to a significant loss of information from the original data and may consequently lead to increased bias (Foss and Markatou, 2018).”

So the problem is not merely interpretability. Nominal categories have no arithmetic mean. A dissimilarity inherited accidentally from an encoding is still a dissimilarity somebody chose. The same paper states why the honest version is hard: “Cluster analysis of mixed-type data sets can be a particularly challenging task because it requires to weigh and aggregate different variables against each other (Hennig and Liao, 2013)”. Foss and Markatou had framed the field the same way in 2018, in the Journal of Statistical Software — as a problem of equitable combination, not of encoding.

Mixed data contains hidden weighting decisions. In each of these five cases the decision has already been made — by a default, a convention, or a column count.

Different field types support different comparisons and transformations. Each row below is a declaration somebody has to make before a distance is computed.

Continuous fields carry magnitude in their differences and usually need scaling. Ordinal fields carry order without spacing. The spacing is supplied from outside the data, by whichever convention you pick — Kaufman and Rousseeuw's rescaled level index, or Podani's tie-corrected ranks. Nominal fields differ without ordering and have no arithmetic average to update. Asymmetric binary fields are the ones where presence means more than shared absence. Declared as such, a pair of zeros sets the indicator δ to 0 and the variable is dropped from that pair's average, which turns the coefficient into the Jaccard index. Left undeclared, the two zeros count as agreement. Counts need interpreting against the time, opportunity, or population that generated them. Nothing in the data announces which row a column belongs to.

Each method embeds a different compromise between type-awareness and scalability. One of those compromises has been measured rather than argued. Costa and colleagues ran a full factorial design of 1,620 data scenarios with fifty replications each — 81,000 simulated data sets. They scored eight distance-based methods by Adjusted Rand Index, in a 2023 paper in Advances in Data Analysis and Classification. Gower's dissimilarity followed by PAM finished last of the eight. Its mean ARI was .136, against .366 for KAMILA, .340 for FAMD/K-Means and .336 for K-Prototypes. More than 75% of Gower/PAM's ARI values fell below .20. Their own summary: “KAMILA, K-Prototypes and sequential Factor Analysis and K-Means clustering typically performed better than other methods.” The interpretable exemplars and the missing-value tolerance of a Gower-plus-medoids partition are real. In that benchmark they cost most of the recoverable structure.

K-Prototypes does not escape the negotiation. It names it. Szepannek, documenting the clustMixType package in The R Journal in December 2018, writes: “The trade off between both terms can be controlled by the parameter λ which has to be specified in advance as well as the number of clusters k. For larger values of λ, the impact of the categorical variables increases.” Both endpoints are known. Costa and colleagues describe the same scalar from the other side: “γl is a weight coefficient for categorical variables in the lth cluster; setting it to zero (thus indicating the absence of categorical variables), one can recover the K-Means algorithm”. Huang's own suggestion was the average standard deviation σ of the numeric variables, with values between σ/3 and 2σ/3 used in his applications. clustMixType therefore sets λ from the data by default, with a caution attached to the convenience: “Note that this should be considered a starting point for further analysis; the explicit choice of λ should be done carefully based on the application context.”

K-Modes drops the arithmetic mean entirely. It represents each cluster by observed category modes and counts mismatches, which keeps representatives real but treats every mismatch alike unless extended. A learned embedding buries the same weighting inside a proxy objective. There it can no longer be read off a single parameter at all, and has to be recovered through neighborhood audits.

Move from field semantics to pairwise examples before fitting a global partition. Classify field types. Define exposure. Set block weights rather than letting column count decide. Review contrast pairs that should match and pairs that should separate. Only then compare algorithms with stability checks.

That last step has a published worked instance to copy. Preud'homme and colleagues benchmarked nine ready-to-use R clustering tools for mixed data. Four were model-based: Kamila, Latent Class Analysis, Latent Class Model, Clustering by Mixture Modeling. Five were distance-based: Gower or Unsupervised Extra Trees dissimilarity followed by hierarchical clustering or PAM, plus K-prototypes. They scored all nine by Adjusted Rand Index on 1,000 generated virtual populations across 7 scenarios. Then they carried every method through to real data — the EPHESUS randomized trial, “conducted on 6632 patients having a recent acute Myocardial Infarction (MI) and a Left Ventricular Ejection Fraction (LVEF) lower than 40%”. Their result, published in Scientific Reports in February 2021: “The simulations revealed the dominance of K-prototypes, Kamila and LCM models over all other methods.” Costa and colleagues, reviewing that study in their related-work section, record the same ordering from outside: “K-Prototypes was the only efficient distance-based method, outperforming all other techniques for larger numbers of clusters”.

Notice what the design supplies that a single fit cannot: a stated metric, many generated populations rather than one dataset, seven varied scenarios, and a named cohort to carry the surviving methods into.

The final groups reflect decisions about which differences matter and how much: the λ you fixed, the variable weights you left at 1, the two-valued column you never declared asymmetric, the ordinal convention you inherited from a package default. Those decisions can affect people, offers, eligibility, or resource allocation.

Document type handling and weights alongside the cluster model. A technically valid dissimilarity can still encode an indefensible operational policy. Foss and Markatou framed the problem in 2018, in the Journal of Statistical Software. Their subject is “the challenge of equitably combining continuous (quantitative) and categorical (qualitative) variables for the purpose of cluster analysis”. Their verdict on existing methods is blunt: “Existing techniques require strong parametric assumptions, or difficult-to-specify tuning parameters.” Eight years of benchmarks later, the tuning parameters are better measured. They are no less yours to justify.