# On the Robustness of Tabular Foundation Models: Test-Time Attacks and In-Context Defenses

arXiv: https://arxiv.org/abs/2506.02978 · HTML: https://mlsanity.com/paper/2506.02978/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 46 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Builds on (most cited first, 30 of 31)

- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [The Limitations of Deep Learning in Adversarial Settings](https://mlsanity.com/paper/1511.07528/) (2015)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [RobustBench: a standardized adversarial robustness benchmark](https://mlsanity.com/paper/2010.09670/) (2020)
- [TabNet: Attentive Interpretable Tabular Learning](https://mlsanity.com/paper/1908.07442/) (2019)
- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [On Evaluating Adversarial Robustness](https://mlsanity.com/paper/1902.06705/) (2019)
- [TabTransformer: Tabular Data Modeling Using Contextual Embeddings](https://mlsanity.com/paper/2012.06678/) (2020)
- [TabICL: A Tabular Foundation Model for In-Context Learning on Large Data](https://mlsanity.com/paper/2502.05564/) (2025)
- [On Evaluating Adversarial Robustness of Large Vision-Language Models](https://mlsanity.com/paper/2305.16934/) (2023)
- [The Lipschitz Constant of Self-Attention](https://mlsanity.com/paper/2006.04710/) (2020)
- [Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learning](https://mlsanity.com/paper/2401.05949/) (2024)
- [TabPFN Unleashed: A Scalable and Effective Solution to Tabular Classification Problems](https://mlsanity.com/paper/2502.02527/) (2025)
- [Representation Learning for Tabular Data: A Comprehensive Survey](https://mlsanity.com/paper/2504.16109/) (2025)
- [Imperceptible Adversarial Attacks on Tabular Data](https://mlsanity.com/paper/1911.03274/) (2019)
- [Prompt Optimization via Adversarial In-Context Learning](https://mlsanity.com/paper/2312.02614/) (2023)
- [Drift-Resilient TabPFN: In-Context Learning Temporal Distribution Shifts on Tabular Data](https://mlsanity.com/paper/2411.10634/) (2024)
- [Fine-tuned In-Context Learning Transformers are Excellent Tabular Data Classifiers](https://mlsanity.com/paper/2405.13396/) (2024)
- [FENCE: Feasible Evasion Attacks on Neural Networks in Constrained Environments](https://mlsanity.com/paper/1909.10480/) (2019)
- [A Unified Framework for Adversarial Attack and Defense in Constrained Feature Space](https://mlsanity.com/paper/2112.01156/) (2021)
- [Closer Look at the Transferability of Adversarial Examples: How They Fool Different Models Differently](https://mlsanity.com/paper/2112.14337/) (2021)
- [Towards Benchmark Datasets for Machine Learning Based Website Phishing Detection: An experimental study](https://mlsanity.com/paper/2010.12847/) (2020)
- [On The Empirical Effectiveness of Unrealistic Adversarial Hardening Against Realistic Adversarial Attacks](https://mlsanity.com/paper/2202.03277/) (2022)
- [Constrained Adaptive Attack: Effective Adversarial Attack Against Deep Neural Networks for Tabular Data](https://mlsanity.com/paper/2406.00775/) (2024)
- [TabularBench: Benchmarking Adversarial Robustness for Tabular Deep Learning in Real-world Use-cases](https://mlsanity.com/paper/2408.07579/) (2024)
- [Zero-shot Meta-learning for Tabular Prediction Tasks with Adversarially Pre-trained Transformer](https://mlsanity.com/paper/2502.04573/) (2025)
- [CaFA: Cost-aware, Feasible Attacks With Database Constraints Against Neural Tabular Classifiers](https://mlsanity.com/paper/2501.10013/) (2025)
- [Evaluating and Safeguarding the Adversarial Robustness of Retrieval-Based In-Context Learning](https://mlsanity.com/paper/2405.15984/) (2024)

## Built on by (most cited first, 3 of 3)

- [DCD-PFN: A Decoupling-Aware Foundation Model for Causal Discovery](https://mlsanity.com/paper/2606.21212/) (2026)
- [Probing Memorization of Tabular In-Context Learning](https://mlsanity.com/paper/2606.31208/) (2026)
- [Boundary-Seeking GAN-Augmented TabTransformer for Adversarially Robust Intrusion Detection](https://mlsanity.com/paper/2607.16348/) (2026)

## Abstract (arXiv)

Recent tabular Foundational Models (FM) such as TabPFN and TabICL, leverage in-context learning to achieve strong performance without gradient updates or fine-tuning. However, their robustness to adversarial manipulation remains largely unexplored. In this work, we present a comprehensive study of the adversarial vulnerabilities of tabular FM, focusing on both their fragility to targeted test-time attacks and their potential misuse as adversarial tools. We show on three benchmarks in finance, cybersecurity and healthcare, that small, structured perturbations to test inputs can significantly degrade prediction accuracy, even when training context remain fixed. Additionally, we demonstrate that tabular FM can be repurposed to generate transferable evasion to conventional models such as random forests and XGBoost, and on a lesser extent to deep tabular models. To improve tabular FM, we formulate the robustification problem as an optimization of the weights (adversarial fine-tuning), or the context (adversarial in-context learning). We introduce an in-context adversarial training strategy that incrementally replaces the context with adversarial perturbed instances, without updating model weights. Our approach improves robustness across multiple tabular benchmarks. Together, these findings position tabular FM as both a target and a source of adversarial threats, highlighting the urgent need for robust training and evaluation practices in this emerging paradigm.