mlsanityData to 14 Oct 2026

Position: Foundation Models for Tabular Data within Systemic Contexts Need Grounding

arXiv 2505.19825 ↗First posted 26 May 2025cs.LG cs.AI cs.DBcited by 6

Not on the map. Why?

Abstract (arXiv)

This position paper argues that foundation models for tabular data face inherent limitations when isolated from operational context - the procedural logic, declarative rules, and domain knowledge that define how data is created and governed. Current approaches focus on single-table generalization or schema-level relationships, fundamentally missing the operational knowledge that gives data meaning. We introduce Semantically Linked Tables (SLT) and Foundation Models for SLT (FMSLT) as a new model class that grounds tabular data in its operational context. We propose dual-phase training: pre-training on open-source code-data pairs and synthetic systems to learn business logic mechanics, followed by zero-shot inference on proprietary data. We introduce the ``Operational Turing Test'' benchmark and argue that operational grounding is essential for autonomous agents in complex data environments.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 126 of its 198 references that are in the map. First posted on arXiv 26 May 2025.

Builds on

126 of 198 references.

References · 126

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  7. 2017Inductive Representation Learning on Large Graphscited by 6,320
  8. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  9. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  10. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  11. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  12. 2018World Modelscited by 1,513
38 more
  1. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  2. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  3. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  4. 2023Mastering Diverse Domains through World Modelscited by 1,253
  5. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  6. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  7. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  8. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 919
  9. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 886
  10. 2021Revisiting Deep Learning Models for Tabular Datacited by 740
  11. 2019Modeling Tabular data using Conditional GANcited by 699
  12. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 688
  13. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  14. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 654
  15. 2022PAL: Program-aided Language Modelscited by 652
  16. 2020The Future of Digital Health with Federated Learningcited by 633
  17. 2020Knowledge Graphscited by 621
  18. 2021Tabular Data: Deep Learning is Not All You Needcited by 535
  19. 2021Deep Neural Networks and Tabular Data: A Surveycited by 491
  20. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 431
  21. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 385
  22. 2022TabDDPM: Modelling Tabular Data with Diffusion Modelscited by 377
  23. 2022The Stack: 3 TB of permissively licensed source codecited by 359
  24. 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Taskscited by 326
  25. 2022Language Models are Realistic Tabular Data Generatorscited by 294
  26. 2025TabICL: A Tabular Foundation Model for In-Context Learning on Large Datacited by 289
  27. 2024CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challengescited by 282
  28. 2020Neurosymbolic AI: The 3rd Wavecited by 255
  29. 2022Binding Language Models in Symbolic Languagescited by 228
  30. 2019Contrastive Learning of Structured World Modelscited by 209
  31. 2023Continual Pre-training of Language Modelscited by 198
  32. 2017Neural-Symbolic Learning and Reasoning: A Survey and Interpretationcited by 196
  33. 2024Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understandingcited by 188
  34. 2023Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Spacecited by 186
  35. 2023Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoningcited by 178
  36. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 166
  37. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 159
  38. 2021SCARF: Self-Supervised Contrastive Learning using Random Feature Corruptioncited by 156

50 most cited of 126 All 126 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Foundation Models for Credit Risk Prediction: A Game Changer?cited by 3
  2. 2026Statistically Indistinguishable, Operationally Distinct: A Formal Barrier for Tabular Foundation Modelscited by 1
  3. 2026Exploring Differences Between Tabular Enterprise Data and Public Benchmarkscited by 1
  4. 2026SALT-KG: A Benchmark for Semantics-Aware Learning on Enterprise Tablescited by 0
  5. 2026Position: Avoid Overstretching LLMs for every Enterprise Taskcited by 0
  6. 2026Relational Learning as World Modelingcited by 0