mlsanityData to 14 Oct 2026

Causal Pre-training Under the Fairness Lens: An Empirical Study of TabPFN

arXiv 2601.17912 ↗First posted 25 Jan 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Foundation models for tabular data, such as the Tabular Prior-data Fitted Network (TabPFN), are pre-trained on a massive number of synthetic datasets generated by structural causal models (SCM). They leverage in-context learning to offer high predictive accuracy in real-world tasks. However, the fairness properties of these foundational models, which incorporate ideas from causal reasoning during pre-training, remain underexplored. In this work, we conduct a comprehensive empirical evaluation of TabPFN and its fine-tuned variants, assessing predictive performance, fairness, and robustness across varying dataset sizes and distributional shifts. Our results reveal that while TabPFN achieves stronger predictive accuracy compared to baselines and exhibits robustness to spurious correlations, improvements in fairness are moderate and inconsistent, particularly under missing-not-at-random (MNAR) covariate shifts. These findings suggest that the causal pre-training in TabPFN is helpful but insufficient for algorithmic fairness, highlighting implications for deploying TabPFN (and similar) models in practice and the need for further fairness interventions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 24 references that are in the map. First posted on arXiv 25 Jan 2026.

Builds on

15 of 24 references.

References · 15

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  3. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
  4. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  5. 2016Fair prediction with disparate impact: A study of bias in recidivism prediction instrumentscited by 672
  6. 2016Inherent Trade-Offs in the Fair Determination of Risk Scorescited by 586
  7. 2018An introduction to domain adaptation and transfer learningcited by 112
  8. 2021Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Testscited by 77
  9. 2018Causal Reasoning for Algorithmic Fairnesscited by 51
  10. 2024Drift-Resilient TabPFN: In-Context Learning Temporal Distribution Shifts on Tabular Datacited by 25
  11. 2024Supervised Algorithmic Fairness in Distribution Shifts: A Surveycited by 24
  12. 2025FairPFN: A Tabular Foundation Model for Causal Fairnesscited by 9
3 more
  1. 2025Towards Fair In-Context Learning with Tabular Foundation Modelscited by 8
  2. 2025Counterfactual Fairness Evaluation of Machine Learning Models on Educational Datasetscited by 3
  3. 2025Can Large Models Teach Student Models to Solve Mathematical Problems Like Human Beings? A Reasoning Distillation Method via Multi-LoRA Interactioncited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025CausalPre: Scalable and Effective Data Pre-Processing for Causal Fairness
  2. 2026TabCausal: Pretraining Across Causal Environments for Tabular Causal Discovery
  3. 2026Improving TabPFN's Synthetic Data Generation by Integrating Causal Structure
  4. 2024Rethinking Fair Representation Learning for Performance-Sensitive Tasks
  5. 2026TabSCM: A practical Framework for Generating Realistic Tabular Data
  6. 2026One Example Is Enough to Pass Fairness Benchmarks: Rethinking Fairness Evaluation for Aligned LLMs
  7. 2026FairDiffuseVQVAE: Sampling-Time Fairness in Tabular Diffusion via Conditional Refinement of Vector-Quantized Latents
  8. 2024Improving LLM Group Fairness on Tabular Data via In-Context Learning