mlsanityData to 14 Oct 2026

TabFM-Auto: Self-Evolving Pipelines for Tabular Foundation Models

arXiv 2609.37989 ↗First posted 29 Sep 2026cs.LGcited by 0

On the map: Synthetic data and self-improvement for LLMs → Privacy-preserving tabular data synthesis

Abstract (arXiv)

Tabular foundation models achieve strong zero-shot accuracy on structured data by pretraining on synthetic tables, but they ignore the column names, task descriptions, and auxiliary files that carry dataset semantics. Meanwhile, self-evolving machine learning engineering (MLE) agents train models from scratch on each dataset, yet jointly searching over features, architectures, and hyperparameters is noisy and prone to overfitting. We introduce TabFM-Auto, which pairs a tabular foundation model, TabFM, with a language model agent that evolves the data pipeline around it. Guided by dataset metadata and validation feedback, TabFM-Auto iteratively refines data cleaning, feature engineering, context selection, and post-processing to reduce TabFM's error. Across all 51 datasets of the TabArena benchmark, five TabFM-Auto configurations with different agents and language models take the top five overall positions, and the best raises TabFM from 1785 to 2013 Elo. The discovered pipelines also transfer to other frozen tabular foundation models (+69 to +143 Elo) with no further search. On the 8 tabular competitions of MLE-Bench, TabFM-Auto ranks first overall among MLE agents.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Privacy-preserving tabular data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 34 of its 46 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

34 of 46 references.

References · 34

  1. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  2. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 867
  3. 2021Revisiting Deep Learning Models for Tabular Datacited by 740
  4. 2022Transformers learn in-context by gradient descentcited by 732
  5. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 715
  6. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  7. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 469
  8. 2020AutoGluon-Tabular: Robust and Accurate AutoML for Structured Datacited by 383
  9. 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentationcited by 366
  10. 2021Transformers Can Do Bayesian Inferencecited by 360
  11. 2025TabICL: A Tabular Foundation Model for In-Context Learning on Large Datacited by 289
  12. 2022TabLLM: Few-shot Classification of Tabular Data with Large Language Modelscited by 277
22 more
  1. 2025AIDE: AI-Driven Exploration in the Space of Codecited by 226
  2. 2023Transformers as Algorithms: Generalization and Stability in In-context Learningcited by 222
  3. 2025TabArena: A Living Benchmark for Machine Learning on Tabular Datacited by 176
  4. 2026TabICLv2: A better, faster, scalable, and open tabular foundation modelcited by 144
  5. 2023Large Language Models for Automated Data Science: Introducing CAAFE for Context-Aware Automated Feature Engineeringcited by 127
  6. 2024TabDPT: Scaling Tabular Foundation Models on Real Datacited by 113
  7. 2024TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensemblingcited by 113
  8. 2024Large Language Models Can Automatically Engineer Features for Few-Shot Tabular Learningcited by 81
  9. 2024CARTE: Pretraining and Transfer for Tabular Learningcited by 76
  10. 2024Making Pre-trained Language Models Great on Tabular Predictioncited by 57
  11. 2023Transformers Learn to Achieve Second-Order Convergence Rates for In-Context Linear Regressioncited by 49
  12. 2024Optimized Feature Generation for Tabular Data via LLMs with Decision Tree Reasoningcited by 49
  13. 2025R&D-Agent: An LLM-Agent Framework Towards Autonomous Data Sciencecited by 41
  14. 2025Real-TabPFN: Improving Tabular Foundation Models via Continued Pre-training With Real-World Datacited by 41
  15. 2024Pre-trained Large Language Models Use Fourier Features to Compute Additioncited by 39
  16. 2023AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Riskscited by 35
  17. 2026MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discoverycited by 28
  18. 2025FoNE: Precise Single-Token Number Embeddings via Fourier Featurescited by 22
  19. 2026Convergent Evolution: How Different Language Models Learn Similar Number Representationscited by 6
  20. 2026TabDPT-Turbo: Efficient In-Context Learning for Tabular Predictioncited by 6
  21. 2026Self-Improving Language Models with Bidirectional Evolutionary Searchcited by 4
  22. 2026Towards Pretraining Text Encoders for TabPFNcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Test-Time Compute for Tabular Foundation Models: Mechanisms, Gains, and Limits
  2. 2026SkillTFM: Gated Skill Evolution for Training-Free Adaptation of Tabular Foundation Models
  3. 2026TabGift: A Context-Efficient Foundation Model for Tabular Data Generation
  4. 2026Mind the Gap? A Distributional Comparison of Real and Synthetic Priors for Tabular Foundation Models
  5. 2026Model-Aware Data Cleaning for Tabular Foundation Models
  6. 2026TabFORGE: A Tabular Foundation Model for Generative Modelling
  7. 2026Xiaomi-TabLDM: A Tabular Foundation Model Technical Report
  8. 2025On Finetuning Tabular Foundation Models