Dataset Inference for Data Provenance and Privacy Auditing in Tabular Foundation Models

AI summary

Overview

Research area: Privacy auditing and data provenance for tabular foundation models (TFMs) — specifically, dataset inference (DI), the task of deciding whether an entire dataset was part of a model's pre-training corpus.

Technical level: Intermediate. The paper assumes familiarity with membership inference, in-context learning, and standard classification metrics (ROCAUC), but its core logic is explained clearly enough for readers with a general machine-learning background.

Scope: The paper proposes and empirically evaluates the first dataset inference method tailored to TFMs, testing it on four state-of-the-art models across external (data-owner) and internal (model-developer) audit settings.

What This Paper Is About

TFMs are increasingly served as black-box APIs and trained on large collections of real-world tabular datasets, which raises the question of whether a given private or proprietary dataset was absorbed into a model's pre-training corpus without permission. No dedicated method currently exists to answer that question for TFMs, and classical point-level membership inference signals (such as per-dataset average loss) do not separate member from non-member datasets. The paper builds a dataset-level inference method that combines many TFM-specific input manipulations and output signals into a single fingerprint per dataset, then uses that fingerprint to audit membership.

Key Contributions

1. Formulates the DI task for TFMs and designs TFM-specific membership signals built from a (manipulation, signal, reduction) triple structure, exploiting the auditor's full control over the context dataset and query points in the in-context learning interface.

2. Shows that individual signals are weak but multi-signal inference works, achieving up to 0.99 ROCAUC for identifying pre-training datasets in some state-of-the-art TFMs trained on real tabular data.

3. Analyzes what drives detectability, identifying pre-training data composition as the main driver: real and real-derived synthetic datasets leave strong dataset-level traces, model size alone has little effect, and prior-synthetic data leaks only when very few datasets are used.

4. Demonstrates practical auditing utility for detecting privacy leakage and unauthorized use of proprietary datasets, with separate instantiations for external audits (data provenance) and internal audits (privacy leakage), plus "blind baseline" controls that separate true membership leakage from mere distribution shift.

Main Findings

• Classical single signals fail. Per-dataset average loss does not clearly separate members from non-members of SAP-RPT-1-OSS (Figure 2); the authors attribute this to TFMs seeing millions of datasets under subsampling, shuffling, and task redefinition, so any single readout captures only a faint trace.

• External audits detect leakage in real-data TFMs. Using an Isolation Forest fit on known non-member fingerprints, SAP-RPT-1-OSS reaches ROCAUC 0.652 ± 0.041 versus 0.442 ± 0.013 on the blind TFM (excess 0.210 ± 0.045), and TabDPT reaches 0.583 ± 0.060 versus 0.472 ± 0.054 (excess 0.110 ± 0.043).

• No external-audit signal for the other two TFMs. nanoTabPFN scores 0.502 ± 0.011 versus 0.502 ± 0.003 on the blind baseline (excess 0.000 ± 0.010), and Real-TabPFN-2.5 scores 0.566 ± 0.036 versus 0.581 ± 0.047 (excess −0.015 ± 0.020).

• Internal audits reach near-perfect detection on SAP-RPT-1-OSS. With leave-one-out cross-validation, logistic regression gives 0.972 ± 0.001 (blind 0.714 ± 0.005), XGBoost 0.992 ± 0.001 (blind 0.697 ± 0.004), and TabPFN-2.5 as the membership classifier 0.994 ± 0.000 (blind 0.689 ± 0.005), for an excess of +0.306 ± 0.005 in the best case.

• TabDPT leaks less. Internal-audit results are 0.675 ± 0.036 (LR), 0.722 ± 0.034 (XGB), and 0.695 ± 0.035 (TabPFN-2.5), against blind baselines of 0.686, 0.662, and 0.611 respectively; only XGBoost and TabPFN-2.5 show a positive excess (+0.060 and +0.084).

• nanoTabPFN and Real-TabPFN-2.5 sit near random under internal auditing (AUC approximately 0.48–0.59 for both the target and blind models), which the authors interpret as little to no retained membership information rather than a failure of their method.

• Blind baselines can themselves score non-trivially, which the authors argue makes excess over the blind baseline the correct measure of true membership leakage rather than raw AUC.

• Task-changing manipulations dominate feature importance. Nine of the ten top SHAP features for SAP-RPT-1-OSS come from task swap (M10) and retrieval (M19); the two strongest are TaskSwap_Wasserstein_Range (mean absolute SHAP 0.8942 on the target versus 0.0000 on the blind model) and TaskSwap_KLD_Range (0.7583 versus 0.0159).

• Distributional distance signals are the most informative readouts. KL divergence (S12) and Wasserstein distance (S14) between predicted and true label distributions dominate the signal axis, because they are small when the model outputs correct label marginals for a dataset.

• Blind baselines rely on different features. Low correlation between target and blind feature-contribution rankings supports the claim that true membership signals are distinct from distributional signals.

• Pre-training data composition matters more than scale. SAP-RPT-1-OSS (trained on 2.18M tables) leaks far more than TabDPT (trained on 123 datasets with SSL-style task generation that swaps the label column and shuffles or drops other columns), which the authors suggest obfuscates membership information.

• Reducing synthetic pre-training data only helps at the extreme. Cutting nanoTabPFN's pre-training corpus from 50,000 to 100 synthetic datasets raises DI ROCAUC to 0.72; the synthetic datasets have roughly 50 rows and the loss plateaus quickly, so no overfitting in the traditional sense occurs.

• Model-capacity and real-versus-synthetic ablations are not reported in the available text. The paper describes the capacity sweep (layers {6, 8, 18}, embedding dimensionality {192, 384, 768}, attention heads {6, 8, 12}, MLP hidden dimensions {768, 1536, 3072}), but the content provided truncates before those results appear; the abstract-level claim is that model size alone has little effect.

Methodology in Plain English

The auditor treats each dataset as a single unit of membership rather than looking at individual rows, because TFM pre-training batches at the dataset level. For every dataset — the suspect one and a set of "known" datasets — the auditor builds a numerical fingerprint by querying the black-box model many times with deliberately altered inputs.

Each feature in the fingerprint comes from three choices. A manipulation reshapes the inputs: for example, "Query only" (M3) throws away the context and feeds only the query features, so a model that has seen the dataset may still predict correctly while a model that has not will guess; "Brainwash" (M17) copies query rows into the context with corrupted labels, so only parametric knowledge retained from pre-training can override the bad context. A signal is a scalar read off the model's output distribution — 14 are used, including classic confidence, margin, and loss measures. A reduction collapses the signal curve produced as a manipulation's parameter is varied into one number, using slopes, area under the curve, means, standard deviations, ranges, consistency, and rank correlation (11 reductions). Combining the three axes yields 678 features per dataset.

For external audits, the auditor only knows non-member datasets, so an Isolation Forest is fit on their fingerprints and the suspect dataset is scored as a possible anomaly. Features are pre-filtered by splitting the known set in half and keeping only those whose suspect-versus-known separation exceeds the 95th percentile of that baseline. Isolation Forest scores are converted into conformal p-values, p(s_sus) = (1 + Σ 1[s_i ≥ s_sus]) / (n + 1), which can be thresholded.

For internal audits, the auditor knows ground-truth membership labels for the known datasets, so a binary classifier (logistic regression, XGBoost, or TabPFN-2.5 used in-context) is trained and evaluated with leave-one-out cross-validation. Features are filtered per fold by mean absolute SHAP value, keeping the top K where K maximizes validation ROCAUC divided by log₂(K+1).

Crucially, every experiment is repeated against a blind baseline: the same datasets and labels, but a different TFM (TabPFN-2.5) that shares no member datasets with the targets. Any accuracy the pipeline achieves on the blind model reflects distribution shift between member and non-member data, not leakage, so only the excess over the blind baseline counts as evidence.

Why This Matters

The work establishes that TFMs can retain dataset-level traces of their pre-training corpora, and that a black-box auditor with no access to weights can extract those traces. It gives both data owners and model developers an empirical tool where none existed, and it introduces blind-baseline controls that guard against the common failure mode of mistaking distribution shift for leakage.

Real-world applications:

• Data owners verifying unauthorized use. A hospital, bank, or retailer holding a proprietary table can test whether a commercial TFM API appears to have trained on it.
• Pre-release privacy audits. Organizations fine-tuning TFMs on internal data can estimate worst-case leakage through their API before deployment.
• Licensing and provenance disputes. The method offers quantitative evidence in disagreements over whether scraped or licensed tabular data entered a model's corpus.
• Procurement and vendor due diligence. Buyers of TFM services can require or perform dataset inference as part of assurance.

Industry relevance: Domains where tabular data is prevalent — finance, e-commerce, and healthcare, as the paper names them — are also the domains where that data is most sensitive, and TFMs are increasingly commercialized as black-box APIs (the paper cites Prior Labs, Fundamental, Kumo AI, and Neuralk-AI). The finding that real-data pre-training leaves far stronger traces than synthetic pre-training is directly relevant to vendors deciding how to curate corpora and to regulators assessing leakage risk.

Future Directions

• Distinguishing "no leakage" from "not detectable." The authors cannot fully rule out that nanoTabPFN and Real-TabPFN-2.5 retain membership information their method fails to extract; stronger or complementary probes would settle this.
• Better attribution of the drivers of detectability. The capacity and real-versus-synthetic ablations are described but their results are not present in the available text, leaving open which architectural or data factors most reduce leakage.
• Extending beyond fixed-corpus models. The blind baseline (TabPFN-2.5) draws fresh synthetic data each batch and shares no fixed members, so the framework's behavior for models without a fixed pre-training corpus is not established.
• Scaling the audit. The method requires many model queries per dataset across 678 features, so reducing query cost and understanding how audit power scales with the number of known datasets are natural next steps.

Target Audience

Researchers working on privacy auditing, membership and dataset inference, and data provenance; TFM developers who need to assess leakage before releasing an API; and legal, compliance, and data-governance teams evaluating whether proprietary tabular data may have been used without authorization. Readers without a machine-learning background will find the problem framing accessible, but the experimental sections assume comfort with ROCAUC, SHAP values, and conformal p-values.

Authors’ abstract

Tabular foundation models (TFMs) are increasingly deployed through black-box APIs and trained on large collections of real-world tabular datasets. As this training paradigm becomes more widely adopted, private or proprietary datasets may be incorporated into pre-training corpora, yet there are currently no dedicated methods for determining whether a given tabular dataset was used to train a TFM. As a solution, we introduce the first dataset inference method for TFMs, aiming to infer whether a suspect dataset was part of a model's pre-training data. We systematically analyze a broad collection of candidate signals that can be observed from a black-box TFM via input manipulation and find that individual signals provide only weak evidence of training data membership. However, by combining multiple complementary signals, we can infer dataset membership for several state-of-the-art TFMs trained on real tabular data, achieving up to 0.99 ROCAUC dependent on the TFM. We then study factors that influence dataset identification, including pre-training data composition, model capacity, and the use of real vs. synthetic data. Our results show that our dataset inference method is a practical auditing tool for detecting privacy leakage and the use of proprietary datasets in TFMs.