Cascaded Flow Matching for Heterogeneous Tabular Data with Mixed-Type Features

AI summary

Overview

Research area: Generative modeling for heterogeneous tabular data — specifically diffusion and flow-matching models that must produce categorical, continuous, and mixed-type (part discrete point mass, part continuous density) features within a single table.

Technical level: Intermediate. The paper assumes familiarity with diffusion models, probability paths, vector fields, and flow matching objectives, though the core idea is explained intuitively.

Scope in one sentence: The paper proposes TabCascade, a two-stage ("cascaded") flow-matching framework that first generates a coarse categorical representation of a data row — including explicit states for missing and inflated values — and then generates the fine-grained numerical details conditioned on that coarse signal.

Metadata: By Markus Mueller, Kathrin Gruber, and Dennis Fok (Econometric Institute, Erasmus University Rotterdam). arXiv:2601.22816v3 [cs.LG], 13 May 2026; published 30 January 2026; license CC BY 4.0. Code: https://github.com/muellermarkus/tabcascade. Keywords: generative model, flow matching, tabular data.

What This Paper Is About

Existing generative models for tabular data train a single shared objective across categorical and numerical features, even though these feature types have fundamentally different structure (discrete probability mass versus continuous density). This causes implicit feature reweighting, where some features dominate training, and it leaves mixed-type features — variables combining discrete states such as missing or zero-inflated values with an otherwise continuous distribution — without dedicated treatment. TabCascade's goal is to learn the joint distribution of categorical and numerical features by splitting the task into an easier low-resolution stage and a harder high-resolution stage, so that discrete states and numerical detail are generated by mechanisms suited to each.

Key Contributions

1. The first cascaded diffusion model for tabular data, and, to the authors' knowledge, the first diffusion model to explicitly address mixed-type feature generation. The framework generalizes to arbitrary and multiple discrete states, not just a single inflated value.
2. A novel cascaded flow-matching framework that decomposes generation into low- and high-resolution parts, using a guided conditional probability path to model high-resolution numerical detail and data-dependent coupling to reduce transport costs. The authors formally prove (Theorem 1) that the data-dependent coupling derived from a distributional-regression-tree encoder yields a tighter transport cost bound than an independent coupling.
3. Feature-type-tailored models that sidestep type-specific loss balancing, thereby preventing the unintended weighting of features prevalent in prior work — with no tuning of relative loss weights required.
4. State-of-the-art results, reported as a detection score improvement of over 50% (51.9% in the abstract), a Wasserstein distance improvement of 50%, and a machine learning efficiency improvement of 30%.

Main Findings

• Detection score improves by 51.9%. Averaged over datasets and seeds, TabCascade (DT) reaches a detection score of 0.787 ± 0.243, versus 0.518 ± 0.296 for CDTD (the second best, underlined) and 0.478 ± 0.375 for TabDDPM. The remaining baselines score 0.430 ± 0.294 (TabDiff), 0.293 ± 0.191 (ARF), 0.205 ± 0.259 (TVAE), 0.202 ± 0.173 (TabSyn) and 0.078 ± 0.075 (CTGAN).

• Numerical fidelity improves substantially. On Wasserstein distance for numerical features, TabCascade achieves 0.004 ± 0.003 versus 0.009 ± 0.006 for CDTD; on the test-relative version, 0.008 ± 0.008 versus 0.012 ± 0.009. Shape (num) is 0.985 ± 0.006 for TabCascade versus 0.962 ± 0.019 for CDTD.

• Overall shape and trend metrics favor the cascade. Shape is 0.984 ± 0.007 (best) versus 0.970 ± 0.011 for CDTD; Shape (test) is 0.975 ± 0.012 versus 0.964 ± 0.012. Trend is 0.965 ± 0.026 (best) versus 0.956 ± 0.032 for CDTD, and Trend (test) is 0.940 ± 0.034 versus 0.932 ± 0.037.

• Mixed-type trends are captured best. On Trend (mixed), TabCascade reaches 0.946 ± 0.032, ahead of ARF at 0.936 ± 0.031 (underlined), with the diffusion baselines lower (CDTD 0.928 ± 0.042, TabDiff 0.920 ± 0.085, TabDDPM 0.867 ± 0.137, TabSyn 0.867 ± 0.059, TVAE 0.787 ± 0.113, CTGAN 0.723 ± 0.087).

• Machine learning efficiency (MLE) is the best reported. TabCascade scores 0.027 ± 0.022, versus 0.039 ± 0.040 for CDTD (underlined), 0.045 ± 0.027 (TabDiff), 0.065 ± 0.049 (ARF), 0.079 ± 0.072 (TVAE), 0.117 ± 0.069 (CTGAN), 0.312 ± 0.942 (TabDDPM) and 0.342 ± 0.933 (TabSyn).

• Trees remain stronger on purely categorical metrics. ARF wins Shape (cat) at 0.993 ± 0.005 (TabCascade 0.986 ± 0.012, underlined), JSD (cat) at 0.008 ± 0.006 (TabCascade 0.018 ± 0.014, underlined) and JSD (cat, test) at 0.016 ± 0.013 (TabCascade 0.023 ± 0.017, underlined).

• Numerical features are harder to learn than categorical ones. Motivational results averaged over datasets and models show that detection scores computed only on categorical features substantially exceed those computed only on numerical features; a per-model breakdown is given in a figure in the Appendix.

• Loss balancing is a real pitfall. Training CDTD on the adult dataset over a grid of 14 relative loss weights for the average categorical feature loss shows detection score improving as the categorical loss weight increases — evidence that the shared-objective design forces a tuning problem that TabCascade avoids entirely.

• Reported metrics in the truncated table. α-Precision values shown are 0.961 ± 0.030 (ARF), 0.736 ± 0.274 (TVAE), 0.858 ± 0.045 (CTGAN), 0.759 ± 0.282 (TabDDPM) and 0.868 ± 0.159 (TabSyn). The remaining α-Precision entries and any further metrics are not reported in the provided content.

Methodology in Plain English

• Reframing "resolution" for tables. Images have pixels to define resolution; tables do not. The authors define categorical features as the low-resolution view of a data row and numerical features as the high-resolution view. Each numerical feature gets a latent categorical code z, so a row's low-resolution counterpart is x_low = (x_cat, z).

• Two-stage generation. The joint distribution is factorized as a sum over z of p_high(x_num | z, x_cat) · p_low(z, x_cat). Sampling is ancestral: first draw z and x_cat from the low-resolution model, then draw x_num conditioned on both. Because H(x_num | z, x_cat) < H(x_num | x_cat) when the two are not independent, a well-chosen z makes the high-resolution problem easier.

• Handling mixed-type features directly. Missing (c_miss) and inflated (c_infl) states become separate categories inside z. If z^(i) = c_miss, the high-resolution model emits NaN with probability 1; if z^(i) = c_infl, it emits the inflated value; otherwise it generates a continuous number. This means the low-resolution model alone determines missingness and inflatedness, and those instances can be masked out of the high-resolution training loss to free up model capacity.

• A guided probability path. The high-resolution model uses flow matching with an ODE whose vector field is conditioned on x_low. Two innovations: (i) a learnable, monotone, feature-specific time schedule γ_t(x_low) — a fifth-degree polynomial in t with closed-form derivative and boundary conditions γ_0 = 0, γ_1 = 1 — allowing non-linear trajectories; and (ii) data-dependent coupling, where the source sample is drawn as x_0 = μ(z) + σ(z) ⊙ ε, positioning each starting point near its target.

• Where z, μ(z) and σ(z) come from. Deterministic, feature-specific encoders map each numerical value to a single Gaussian component. Two encoder choices are used: a distributional regression tree (DT) and a Gaussian mixture model (GMM); z is the terminal leaf index for the tree, or the argmax of log w_k p_k(x) for the GMM. The component's mean and standard deviation parameterize the source distribution. Near-zero variance is treated as an inflated value; missing values are removed before fitting and re-added afterwards as c_miss.

• Low-resolution model choice. Any categorical generative model can serve as the low-resolution component. The authors use the categorical part of CDTD, chosen for its efficiency with high-cardinality features.

• Experiments. TabCascade is benchmarked against CTGAN, TVAE, ARF, TabDDPM, TabSyn, TabDiff and CDTD, with all models aligned as closely as possible; since no baseline natively supports missing-data generation, each is augmented with an encoding-based missing-value mechanism. ForestDiffusion and SMOTE are excluded, following Mueller et al. (2025), because they are inefficient on medium-to-large datasets — on adult, ForestDiffusion's default hyperparameters require several hours of training, and early stopping is not viable because it estimates separate models per feature and timestep.

Why This Matters

Impact on research. The paper argues that treating categorical and numerical features under one shared objective is a design flaw rather than a convenience, and that mixed-type features deserve their own representational treatment. It supplies both an architectural alternative and a formal result (a tighter transport cost bound) for why conditioning the source distribution on coarse information helps. Because the framework accepts any categorical generative model as its low-resolution stage, it is a modular template rather than a single fixed architecture.

Real-world applications:

• Economic survey data, where a reported working time of zero hours can indicate unemployment rather than a true zero — a participation state the model can generate explicitly.
• Medical data, where a drug dosage of zero may indicate absence of treatment and missing values may reflect reluctance to disclose information; realistic synthesis matters for downstream analysis.
• Imputation and missing-data methods, where downstream models that handle missingness explicitly require data in which missingness is generated as part of the data-generating process.
• Psychological questionnaires, where missing answers may themselves point toward personality traits, making missingness informative signal rather than noise.

Industry relevance. Machine learning efficiency (reported as the lowest value of 0.027 among the compared models) and the removal of relative loss-weight tuning are practical advantages for teams that cannot afford large hyperparameter sweeps on tabular pipelines. The cascaded design also spends capacity where it is needed — on numerical detail — rather than on coarse categorical structure that the paper shows is comparatively easy to learn.

Future Directions

• Generalizing the encoder choice. The main results use the distributional-regression-tree encoder; a Gaussian mixture model is described as an additional experiment with details deferred to Appendix A.5. A systematic comparison of encoder families, and of how leaf or component granularity affects sample quality, remains open.
• Beyond a single blocked-out discrete state. The paper states the setup "trivially extends" to arbitrary mixed-type structures, including multiple inflated values, and its derivations allow for multiple discrete states. Demonstrating this on real datasets with censoring or one-inflation alongside missingness is the natural next test.
• Swapping the low-resolution component. Since any categorical generative model can be plugged in, a question the paper leaves open is how much of the reported gain comes from the cascade itself versus from the specific CDTD categorical component.
• Aligning evaluation. TabCascade leads on detection score, numerical Wasserstein distance, shape, trend and MLE, but ARF still leads on Shape (cat), JSD (cat) and JSD (cat, test). Whether the categorical stage can be strengthened without sacrificing the numerical gains — and whether these metric disagreements reflect genuine quality differences — is unresolved.

Target Audience

Researchers and practitioners in generative modeling and tabular machine learning who already understand diffusion models and flow matching, and who need either a stronger synthesizer for heterogeneous tables or a conceptual template for decomposing generation by feature type. It is also relevant to applied statisticians and data scientists working with survey, medical, or questionnaire data, where zero-inflation, censoring, and informative missingness are routine — though those readers will need to work through the flow-matching machinery to follow the technical sections.

Authors’ abstract

Advances in generative modeling have recently been adapted to tabular data containing discrete and continuous features. However, generating mixed-type features that combine discrete states with an otherwise continuous distribution in a single feature remains challenging. We advance the state-of-the-art in diffusion models for tabular data with a cascaded approach. We first generate a low-resolution version of a tabular data row, that is, the collection of the purely categorical features and a coarse categorical representation of numerical features. Next, this information is leveraged in the high-resolution flow matching model via a novel guided conditional probability path and data-dependent coupling. The low-resolution representation of numerical features explicitly accounts for discrete outcomes, such as missing or inflated values, and therewith enables a more faithful generation of mixed-type features. We formally prove that this cascade tightens the transport cost bound. The results indicate that our model generates significantly more realistic samples and captures distributional details more accurately, for example, the detection score improves by 51.9\%. Code is available at https://github.com/muellermarkus/tabcascade.