mlsanityData to 14 Oct 2026

LimiX-2: A Contextual Mechanism Network Towards General Structured-Data Intelligence

arXiv 2609.17488 ↗First posted 15 Sep 2026cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

We introduce LimiX-2, a new model in the LimiX family, developed through model and data scaling guided by our previously established scaling laws. LimiX-2 adopts the Contextual Mechanism Networks (CMNs) paradigm and is pretrained with Context-Conditional Masked Modeling (CCMM). CMNs shifts the organizing principle of in-context learning from target-centric prediction to mechanism-oriented joint modeling. Rather than centering the network on the p(y mid x, Dcontext) objective of conventional tabular PFNs, it is designed around learning p(x, y mid Dcontext), a context-dependent representation of the joint structure underlying data generation. Pretraining uses synthetic datasets generated by structural causal models (SCMs) spanning diverse graph structures, functional mechanisms, and observation processes. Evaluations on TabArena, TALENT, and BCCO show that LimiX-2 outperforms current dataset-specific models and tabular foundation models. Beyond predictive performance, the CMN paradigm also promotes causal awareness in LimiX-2: its feature attention encodes direct causal relationships, enabling accurate causal skeleton recovery.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 38 of its 63 references that are in the map. First posted on arXiv 15 Sep 2026.

Builds on

38 of 63 references.

References · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  5. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  7. 2020GLU Variants Improve Transformercited by 1,804
  8. 2018World Modelscited by 1,513
  9. 2019Root Mean Square Layer Normalizationcited by 1,480
  10. 2021Revisiting Deep Learning Models for Tabular Datacited by 740
  11. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  12. 2020AutoGluon-Tabular: Robust and Accurate AutoML for Structured Datacited by 383
26 more
  1. 2021Transformers Can Do Bayesian Inferencecited by 360
  2. 2019DAG-GNN: DAG Structure Learning with Graph Neural Networkscited by 331
  3. 2025TabICL: A Tabular Foundation Model for In-Context Learning on Large Datacited by 289
  4. 2018CatBoost: gradient boosting with categorical features supportcited by 208
  5. 2019Learning Sparse Nonparametric DAGscited by 207
  6. 2025TabArena: A Living Benchmark for Machine Learning on Tabular Datacited by 176
  7. 2026TabICLv2: A better, faster, scalable, and open tabular foundation modelcited by 144
  8. 2022Overcoming a Theoretical Limitation of Self-Attentioncited by 113
  9. 2024TabDPT: Scaling Tabular Foundation Models on Real Datacited by 113
  10. 2024TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensemblingcited by 113
  11. 2024Better by Default: Strong Pre-Tuned MLPs and Boosted Trees on Tabular Datacited by 92
  12. 2022Amortized Inference for Causal Structure Learningcited by 90
  13. 2025LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligencecited by 79
  14. 2025Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Modelscited by 77
  15. 2023TabR: Tabular Deep Learning Meets Nearest Neighbors in 2023cited by 74
  16. 2024TALENT: A Tabular Analytics and Learning Toolboxcited by 52
  17. 2025Scalable-Softmax Is Superior for Attentioncited by 45
  18. 2024Revisiting Nearest Neighbor for Tabular Data: A Deep Tabular Baseline Two Decades Latercited by 30
  19. 2026PluRel: Synthetic Data unlocks Scaling Laws for Relational Foundation Modelscited by 19
  20. 2022CIPCaD-Bench: Continuous Industrial Process datasets for benchmarking Causal Discovery methodscited by 10
  21. 2026EXAONE Tabular 1.0 : Technical Reportcited by 10
  22. 2026LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Modelscited by 9
  23. 2026TabCausal: Pretraining Across Causal Environments for Tabular Causal Discoverycited by 4
  24. 2026Mitra-v2 Technical Reportcited by 4
  25. 2020Causal motifs and existence of endogenous cascades in directed networks with application to company defaultscited by 1
  26. 2026MaskTab: Scalable Masked Tabular Pretraining with Scaling Laws and Distillation for Industrial Classificationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026Causilo Technical Reportcited by 3
  2. 2026Message Passing Does More with Less for In-Context Learning on Graphscited by 0
  3. 2026From Behavior to Provenance: Attributing Tabular Foundation Models to Synthetic Pretraining Datacited by 0
  4. 2026When Does Synthetic Relational Data Teach Models to Use Relations? Tracing Predictive Structure from Pretraining Data to Model Behaviorcited by 0
  5. 2026Test-Time Compute for Tabular Foundation Models: Mechanisms, Gains, and Limitscited by 0
  6. 2026When Repeated Measurements Hurt Tabular Foundation Modelscited by 0
  7. 2026TILE: Bridging LLMs and Table Foundation Models for Tabular Learning with Stringscited by 0
  8. 2026OrionMoE-v1.0: A Mixture-of-Experts Foundation Model for Tabular In-Context Learningcited by 0