# FairPFN: Transformers Can do Counterfactual Fairness

arXiv: https://arxiv.org/abs/2407.05732 · HTML: https://mlsanity.com/paper/2407.05732/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 16 references that are in the map.
- First posted on arXiv 8 Jul 2024.

## Builds on (most cited first, 10 of 10)

- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018)
- [Transformers Can Do Bayesian Inference](https://mlsanity.com/paper/2112.10510/) (2021)
- [Causal Discovery with Continuous Additive Noise Models](https://mlsanity.com/paper/1309.6779/) (2013)
- [A Clarification of the Nuances in the Fairness Metrics Landscape](https://mlsanity.com/paper/2106.00467/) (2021)
- [DoWhy: An End-to-End Library for Causal Inference](https://mlsanity.com/paper/2011.04216/) (2020)
- [Causal Fairness Analysis](https://mlsanity.com/paper/2207.11385/) (2022)
- [Learning for Counterfactual Fairness from Observational Data](https://mlsanity.com/paper/2307.08232/) (2023)
- [A Brief Introduction to Causal Inference in Machine Learning](https://mlsanity.com/paper/2405.08793/) (2024)

## Built on by (most cited first, 8 of 8)

- [Do-PFN: In-Context Learning for Causal Effect Estimation](https://mlsanity.com/paper/2506.06039/) (2025)
- [Position: The Future of Bayesian Prediction Is Prior-Fitted](https://mlsanity.com/paper/2505.23947/) (2025)
- [Towards Fair In-Context Learning with Tabular Foundation Models](https://mlsanity.com/paper/2505.09503/) (2025)
- [Does TabPFN Understand Causal Structures?](https://mlsanity.com/paper/2511.07236/) (2025)
- [Sequential Conditional Transport on Probabilistic Graphs for Interpretable Counterfactual Fairness](https://mlsanity.com/paper/2408.03425/) (2024)
- [Amortized Inference of Causal Models via Conditional Fixed-Point Iterations](https://mlsanity.com/paper/2410.06128/) (2024)
- [EquiTabPFN: A Target-Permutation Equivariant Prior Fitted Networks](https://mlsanity.com/paper/2502.06684/) (2025)
- [Fairness in Augmented Graph Learning: A Survey](https://mlsanity.com/paper/2504.21296/) (2025)

## Abstract (arXiv)

Machine Learning systems are increasingly prevalent across healthcare, law enforcement, and finance but often operate on historical data, which may carry biases against certain demographic groups. Causal and counterfactual fairness provides an intuitive way to define fairness that closely aligns with legal standards. Despite its theoretical benefits, counterfactual fairness comes with several practical limitations, largely related to the reliance on domain knowledge and approximate causal discovery techniques in constructing a causal model. In this study, we take a fresh perspective on counterfactually fair prediction, building upon recent work in in context learning (ICL) and prior fitted networks (PFNs) to learn a transformer called FairPFN. This model is pretrained using synthetic fairness data to eliminate the causal effects of protected attributes directly from observational data, removing the requirement of access to the correct causal model in practice. In our experiments, we thoroughly assess the effectiveness of FairPFN in eliminating the causal impact of protected attributes on a series of synthetic case studies and real world datasets. Our findings pave the way for a new and promising research area: transformers for causal and counterfactual fairness.