Hybrid Autoencoders for Tabular Data: Leveraging Model-Based Augmentation in Low-Label Settings

AI summary

Overview

Research area: Machine learning for tabular data — specifically self-supervised and semi-supervised representation learning when labeled examples are scarce.

Technical level: Intermediate. Readers should be comfortable with autoencoders, decision trees, loss functions, and standard evaluation metrics.

Scope: The paper introduces TANDEM, a hybrid autoencoder that pairs a neural network encoder with an oblivious soft decision tree encoder during self-supervised pretraining, so that the tree acts as a training-time "model-based augmentation" for a neural encoder that is used alone at inference.

What This Paper Is About

Deep neural networks often underperform tree-based models such as XGBoost, CatBoost and GBDT on tabular data, partly because neural networks are spectrally biased toward smooth, low-frequency functions while tabular decision boundaries are often sharp and irregular. Self-supervised learning is a natural fix, but tabular augmentations (noise injection, feature swapping) tend to distort categorical features and break feature relationships. The paper's goal is to replace unreliable data augmentation with model augmentation: train two encoders with different inductive biases together so each one's structure enriches the other's representations, then deploy only the neural encoder.

Key Contributions

1. TANDEM architecture: A hybrid self-supervised autoencoder combining a fully connected neural encoder, an ensemble of oblivious soft decision tree (OSDT) encoders, a shared decoder, and sample-specific stochastic gating networks, trained with reconstruction, alignment, and latent-representation-similarity (LRS) losses.
2. Strong low-label results: Demonstrating that the neural encoder's learned representations support both classification and regression under low-label conditions, surpassing established deep learning and tree-based baselines (including TabPFN for classification).
3. Systematic label-budget study: Experiments across a diverse suite of tabular datasets with the number of labeled samples varied from 50 to 1000 per dataset, establishing robustness across low-label regimes.
4. Spectral and gating analysis: Qualitative spectral analysis plus a quantitative comparison of gating activations, showing that the neural and tree encoders capture distinct, complementary inductive biases (smoothing versus high-frequency retention).

Main Findings

• Classification at 400 labels: TANDEM reaches the highest mean accuracy (0.7124) and best mean rank (1.58) across the 19 classification datasets. The closest competitor is TabPFN with mean accuracy 0.7012 and mean rank 2.56; CatBoost follows at 0.6731 (rank 4.16), MLP at 0.6721 (rank 4.84), XGBoost at 0.6706 (rank 4.47), TabM at 0.6532 (rank 4.84), multinomial logistic regression at 0.6380 (rank 6.16), DeepTLF at 0.5626 (rank 8.05) and TabNet at 0.5454 (rank 8.00).
• Regression at 400 labels: TANDEM attains the lowest mean MSE (0.3234) and best mean rank (3.38) across the 13 regression datasets, with XGBoost as the nearest competitor (mean MSE 0.3405, rank 4.15). CatBoost reports mean MSE 0.3318 (rank 4.00), MLP 0.3877, TabM 0.4006, SubTab 0.4354, SCARF 0.4372, TabNet 0.5174, VIME 0.5183 and logistic regression 0.6918.
• Not best on every dataset: The paper describes TANDEM as "competitive or best on most datasets" rather than uniformly best. For example, in classification it is below TabPFN on EL (0.6940 vs 0.7723) and below multinomial logistic regression on BM (0.8233 vs 0.8325); in regression it is above CatBoost on BF (1.0057 vs 0.6565) and above MLP on YP (1.2300 vs 1.0278).
• Dolan–Moré robustness: The Dolan–Moré profiles at 400 labels (Figures 4 and 6) are reported to underscore robustness across datasets for both classification and regression.
• Ablations confirm every component matters: At 400 labels on classification, full TANDEM reaches mean accuracy 0.7124 and mean rank 1.74, versus 0.6815 (rank 4.45) for SS-AE, 0.6941 (rank 3.61) for SS-AE + Gating, 0.6600 (rank 4.71) for the OSDT AE + Gating variant, 0.6966 (rank 2.92) for TANDEM without gating, and 0.6971 (rank 2.79) for TANDEM without the LRS and alignment losses. On regression, TANDEM has mean MSE 0.3235 (rank 1.92), versus 0.3628 (rank 2.69) for SS-AE, 0.6551 for SS-AE + Gating, 0.9207 for OSDT AE + Gating, 0.5397 for TANDEM without gating, and 0.5832 for TANDEM without LRS and alignment.
• Gating removes the most information in the OSDT stream alone: The OSDT AE + Gating ablation is the weakest variant in both tasks (0.6600 classification accuracy; 0.9207 regression MSE), indicating the tree encoder needs the neural partner.
• Consistent across label budgets: Learning curves from 50 to 1000 labels show TANDEM is consistently either the top method or competitive with the best throughout the low-label regime in both classification and regression.
• Gating behaves as an adaptive frequency filter: Using the unnormalized discrete Fourier transform (NUDFT) over the 50 most variant features, computed on the class where TANDEM achieves its highest accuracy, the neural gating transformation reduces high-frequency components more strongly than the tree-based transformation. The neural and tree-based gated views inside TANDEM show the strongest spectral contrast, with the neural gate acting as a strong smoother and the aggregated tree gate retaining more high-frequency content.
• TabPFN coverage caveat: TabPFN supports a maximum of 10 classes and could not be evaluated on the helena dataset (13 classes); its mean and rank therefore exclude that dataset.

Methodology in Plain English

The starting point is a large pool of unlabeled tabular data plus a small labeled set. Each input is first passed through a small neural "gating" network that outputs a per-feature mask in [0, 1] — a soft decision about which features matter for this particular sample. The mask values are produced by adding Gaussian noise of fixed standard deviation 0.5 to the network output and clipping to [0, 1], which nudges values toward binary choices while keeping gradients flowing. The masked input is then fed in parallel to two encoders:

• a standard fully connected neural encoder, and
• an ensemble of oblivious soft decision trees, where all nodes at a given depth share one learned projection vector and threshold. Each tree routes probabilistically through a temperature-scaled sigmoid, producing a soft distribution over its leaves; the leaf probability vectors are averaged across all trees to form the tree latent code.

Crucially, the tree encoder gets its own gating network at every depth, so it can pick different features at different levels of the hierarchy and make progressively refined decisions.

A single decoder is shared by both encoders, so both latent codes are reconstructed back to the input space. Training combines three losses: reconstruction error for each encoder, an alignment loss that pulls the two reconstructions toward each other, and a latent-representation-similarity loss that minimizes the average cosine distance between the two latent codes. This pushes the two encoders toward representations that are consistent but still shaped by their own inductive biases.

After pretraining on the unlabeled pool, a single-layer MLP classifier or regressor is trained on the labeled subset using the neural encoder only. The encoder is frozen for the first 25 epochs and then fine-tuned for 25 more at a reduced learning rate, with early stopping on validation accuracy or MSE. The gating network is frozen during fine-tuning and acts as a per-sample feature selector. At inference, the tree encoder is dropped entirely — only the neural encoder and the light predictor are deployed. The full pretraining run uses 100 epochs, batch size 128, RMSprop as the optimizer, and Optuna with 50 trials for hyperparameter selection. Datasets were filtered to at least 2,500 samples per class, with 2,000 samples per class used for self-supervised pretraining; this yielded 19 classification and 13 regression datasets. Each experiment was repeated 100 times with varying seeds and splits.

Why This Matters

The paper reframes tabular self-supervision: instead of inventing augmentations that fit all datasets, it enriches the model with a second, structurally different encoder that shapes the neural representation during training and then disappears at deployment. That keeps inference cheap and compatible with existing neural pipelines while importing some of the inductive bias that makes trees strong on tabular data.

Real-world applications (domains the paper names explicitly):

• Healthcare: tabular clinical datasets where labels are expensive and scarce while unlabeled records are abundant.
• Finance: tabular risk and credit data where tree ensembles are the incumbent and labeled outcomes are limited.
• Biology: heterogeneous feature tables with few labeled instances per class.
• Recommendation systems: tabular user/item feature contexts where label collection is costly.

Industry relevance: the released design preserves a lean inference path (neural encoder plus a lightweight head), so it can be dropped into existing neural deployment stacks without shipping a tree ensemble. The gating networks double as an interpretable, per-sample feature-selection signal, and the results are positioned as competitive with tuned gradient-boosted trees (XGBoost, CatBoost) that currently dominate applied tabular work with minimal tuning.

Future Directions

1. Integrating the design into transformer-based tabular models. The authors suggest hybrid encoders, sample-specific gating, and model-based augmentation could be incorporated into transformer architectures, especially when combined with efficient large-model optimization techniques such as adaptive low-rank gradient methods and structured sparsification for scalable fine-tuning.
2. Scaling beyond small, low-label evaluations. The paper acknowledges limitations around small sample sizes and its exclusive focus on low-label settings, and frames these small-scale evaluations as a step toward scalable solutions and tabular foundation models.
3. Understanding why the two gating streams diverge. The spectral analysis shows the neural gate smooths more and the tree gate retains more high-frequency content; the exact mechanism linking this contrast to downstream gains remains an open question.
4. Reducing dependence on careful hyperparameter tuning. TANDEM requires tuning across pretraining and fine-tuning (learning rate, encoder depth, weight decay) via Optuna, and whether the hybrid benefits hold under looser tuning budgets is not reported.

Target Audience

Researchers and practitioners working on tabular machine learning, self-supervised and semi-supervised representation learning, or hybrid neural-symbolic architectures. It is most useful to readers who already know how autoencoders and decision trees are trained and want to see how two different inductive biases can be coupled through a shared decoder and sample-specific gating. Applied data scientists in healthcare, finance, and biology who face scarce labels but abundant unlabeled tables will find the low-label benchmark results and the deployment-friendly inference path directly relevant.

Authors’ abstract

Deep neural networks often under-perform on tabular data due to their sensitivity to irrelevant features and a spectral bias toward smooth, low-frequency functions. These limitations hinder their ability to capture the sharp, high-frequency signals that often define tabular structure, especially under limited labeled samples. While self-supervised learning (SSL) offers promise in such settings, it remains challenging in tabular domains due to the lack of effective data augmentations. We propose a hybrid autoencoder that combines a neural encoder with an oblivious soft decision tree (OSDT) encoder, each guided by its own stochastic gating network that performs sample-specific feature selection. Together, these structurally different encoders and model-specific gating networks implement model-based augmentation, producing complementary input views tailored to each architecture. The two encoders, trained with a shared decoder and cross-reconstruction loss, learn distinct yet aligned representations that reflect their respective inductive biases. During training, the OSDT encoder (robust to noise and effective at modeling localized, high-frequency structure) guides the neural encoder toward representations more aligned with tabular data. At inference, only the neural encoder is used, preserving flexibility and SSL compatibility. Spectral analysis highlights the distinct inductive biases of each encoder. Our method achieves consistent gains in low-label classification and regression across diverse tabular datasets, outperforming deep and tree-based supervised baselines.