# Privacy Vulnerabilities of Attention Layers in Tabular Foundation Models and Protection of High-Risk Queries

arXiv: https://arxiv.org/abs/2606.26021 · HTML: https://mlsanity.com/paper/2606.26021/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Membership inference attacks, one of 12 directions in LLM safety and security.
- Builds on 35 of its 50 references that are in the map.
- First posted on arXiv 24 Jun 2026.

## Placed in

- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [What Can Transformers Learn In-Context? A Case Study of Simple Function Classes](https://mlsanity.com/paper/2208.01066/) (2022)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [TabICL: A Tabular Foundation Model for In-Context Learning on Large Data](https://mlsanity.com/paper/2502.05564/) (2025)
- [Enhanced Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/2111.09679/) (2021)
- [TabArena: A Living Benchmark for Machine Learning on Tabular Data](https://mlsanity.com/paper/2506.16791/) (2025)
- [The mechanistic basis of data dependence and abrupt learning in an in-context classification task](https://mlsanity.com/paper/2312.03002/) (2023)
- [TabDPT: Scaling Tabular Foundation Models on Real Data](https://mlsanity.com/paper/2410.18164/) (2024)
- [Low-Cost High-Power Membership Inference Attacks](https://mlsanity.com/paper/2312.03262/) (2023)
- [Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generation](https://mlsanity.com/paper/2309.11765/) (2023)
- [Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Models](https://mlsanity.com/paper/2305.15594/) (2023)
- [Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Models](https://mlsanity.com/paper/2510.21204/) (2025)
- [Privacy-Preserving In-Context Learning for Large Language Models](https://mlsanity.com/paper/2305.01639/) (2023)
- [Membership Inference Attacks Against In-Context Learning](https://mlsanity.com/paper/2409.01380/) (2024)
- [On the Privacy Risk of In-context Learning](https://mlsanity.com/paper/2411.10512/) (2024)
- [Overconfidence is a Dangerous Thing: Mitigating Membership Inference Attacks by Enforcing Less Confident Prediction](https://mlsanity.com/paper/2307.01610/) (2023)
- [Understanding Emergent In-Context Learning from a Kernel Regression Perspective](https://mlsanity.com/paper/2305.12766/) (2023)
- [Private prediction for large-scale synthetic text generation](https://mlsanity.com/paper/2407.12108/) (2024)
- [Context-Aware Membership Inference Attacks against Pre-trained Large Language Models](https://mlsanity.com/paper/2409.13745/) (2024)
- [TabSTAR: A Tabular Foundation Model for Tabular Data with Text Fields](https://mlsanity.com/paper/2505.18125/) (2025)
- [TabFlex: Scaling Tabular Learning to Millions with Linear Attention](https://mlsanity.com/paper/2506.05584/) (2025)
- [DP-TabICL: In-Context Learning with Differentially Private Tabular Data](https://mlsanity.com/paper/2403.05681/) (2024)
- [TabularBench: Benchmarking Adversarial Robustness for Tabular Deep Learning in Real-world Use-cases](https://mlsanity.com/paper/2408.07579/) (2024)
- [LoRA-Leak: Membership Inference Attacks Against LoRA Fine-tuned Language Models](https://mlsanity.com/paper/2507.18302/) (2025)
- [Locally Differentially Private In-Context Learning](https://mlsanity.com/paper/2405.04032/) (2024)
- [In-Context Learning for Extreme Multi-Label Classification](https://mlsanity.com/paper/2401.12178/) (2024)
- [ContextLeak: Auditing Leakage in Private In-Context Learning Methods](https://mlsanity.com/paper/2512.16059/) (2025)

## Built on by (most cited first, 1 of 1)

- [SoK: Rethinking Membership Inference in Tabular Machine Learning](https://mlsanity.com/paper/2610.18486/) (2026)

## Abstract (arXiv)

Tabular foundation models are commonly assumed to present limited privacy concerns as they are often pre-trained on large collections of synthetic data. However, these models leverage in-context learning, where sensitive records may be provided directly at inference time as labelled context examples. In this paper, we demonstrate that predictions generated via the attention mechanism leak sufficient information to enable effective Membership Inference Attacks (MIAs). To highlight this vulnerability, we propose AMIA (Attention-based Membership Inference Attack), a shadow-model-free attack that exploits the concentration of transformer attention patterns. Our results show that attention mechanisms reveal strong membership signals, which exceed classical confidence-based attacks, achieving an average gain of 7.7%, specially in low false-positive regimes. To mitigate this risk, we introduce an inference-time defence inspired by k-anonymity principles. This approach reduces the uniqueness of context-key representations without introducing random noise or retraining the model. By targeting only high-risk queries identified through AMIA scores, the defence substantially reduces membership leakage of this attack by an average of 50% and 25% against confidence-based attacks, while preserving predictive utility with only 3.9% performance degradation. Beyond showing that context examples are vulnerable, we further demonstrate that fine-tuning introduces an additional source of privacy risk. In particular, samples whose prediction confidence increases after fine-tuning become more susceptible to MIAs, indicating that fine-tuning can amplify memorisation and expose sensitive training information through confidence shifts.