The Pulse
TabFM-Auto Takes TabArena’s Top Five Spots
A Google Research-led team says its TabFM-Auto system placed five configurations atop TabArena’s 51-dataset leaderboard. The system uses language-model agents to rewrite data pipelines around a fixed tabular model, raising its Elo rating fr
AI.info Team · Written with AI ·
Five configurations, five top places
Five versions of one AI-assisted system occupy the top five positions in a 51-dataset tabular-machine-learning benchmark, according to a paper submitted to arXiv on September 29. The system, TabFM-Auto, raises its underlying TabFM model’s overall rating from 1,785 to 2,013 Elo by using a language-model agent to improve the data pipeline around the model, rather than retraining the model itself.
The authors write, “On the 8 tabular competitions of MLE-Bench, TabFM-Auto ranks first overall among MLE agents.” Deqing Fu, Huangyuan Su, Rajat Sen, Taman Narayan, Sujay Sanghavi, Abhimanyu Das and Weihao Kong
TabArena’s test set includes 38 classification datasets and 13 regression datasets. The paper compares the five TabFM-Auto configurations against 66 methods, including other tabular foundation models and tuned AutoGluon ensembles; the configurations take the top five overall Elo ratings in the authors’ evaluation. Elo is a relative rating based on pairwise results, not an accuracy percentage.
The agent changes the table, not the model
Tabular foundation models such as TabFM make predictions from numerical values and category codes, but do not directly understand column names or task descriptions. That can leave useful information in a dataset’s schema unused: a model may see a column of numbers without knowing that it represents blood pressure, a physical measurement or a diagnostic code.
TabFM-Auto gives a coding agent access to the metadata and validation results, then asks it to revise four parts of the pipeline: data cleaning, feature engineering, selection of rows used as context, and post-processing of predictions. The 400-million-parameter TabFM model stays frozen throughout the search. The agent evaluates candidate pipelines with three-fold cross-validation on training data, and the paper says a search can run for up to 96 evaluations or six hours on one H100 GPU.
Those changes can range from recoding values that stand for missing data to building domain-specific formulas from column names. On larger or imbalanced datasets, the agent can also select context rows to preserve minority classes and adjust output probabilities to match the data’s class balance. The paper reports that the agents changed the feature table in 97 of 102 runs examined across two configurations.
Strong results, with a costly search
The gains varied by task. On the 13 regression datasets, the best configuration reaches 2,512.9 Elo, 467 points above the base TabFM result; on the 38 classification datasets, the best rating is 1,966.3 Elo, about 198 points above TabFM. The authors also report that the discovered pipelines improved results when transferred to other frozen tabular foundation models, adding between 69 and 143 Elo without another search.
A second evaluation covered eight tabular competitions from MLE-Bench. The researchers gave TabFM-Auto a 24-hour budget per competition and compared it with published submissions from other machine-learning engineering agents. In that comparison, the Claude Code and Opus 5 configuration ranked first, at 1,827 Elo, while the Antigravity and Gemini 3.8 Flash configuration ranked second, at 1,734 Elo. Other agents led on some individual competitions, the paper says, even though the two TabFM-Auto variants topped the aggregate ranking.
That performance required substantial compute and API use. The authors estimate that their five TabArena sweeps cost about $17,600 in language-model API fees at Vertex AI list prices; repeating the search independently across all evaluation folds would raise token and GPU costs by a factor of 16. The system’s leaderboard results therefore show what agent-guided pipeline work can achieve under a large search budget, not that the same gains come cheaply on every new dataset.
Schema knowledge is still the dividing line
The paper’s strongest gains came when column names or task descriptions revealed what the data represented. On 17 datasets with informative schemas, the agents could write formulas tied to the domain; on datasets with anonymous or generic column names, they relied more on statistical and relational transformations, and the reported error reductions were smaller.
The authors identify repeated validation as another limitation: each new dataset requires pipeline search, and the experiments do not eliminate that expense. Their results make a narrower claim than “an AI agent replaces machine-learning engineers.” TabFM-Auto keeps a particular predictor fixed and asks an agent to make that predictor’s inputs and outputs more useful—a strategy that led the tested configurations to the top five TabArena positions.