mlsanityData to 14 Oct 2026

TabPATE: Differentially Private Tabular In-Context Learning Without Public Data

arXiv 2606.31474 ↗First posted 30 Jun 2026cs.LGcited by 0

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Tabular foundation models enable accurate in-context learning (ICL) from small labeled datasets, but the private records placed in context can leak through model predictions. We first show that even basic membership inference attacks succeed against tabular ICL, motivating formal privacy protection. We then introduce TabPATE, a differentially private PATE-style defense for tabular ICL that does not require public in-distribution data. TabPATE partitions the private context across teacher models, privately aggregates their labels on synthetic tabular queries, and releases the resulting labeled queries as a student context. Because tabular features are bounded and relatively low-dimensional, useful queries can be generated from feature ranges alone or from lightly privatized marginals. Across tabular benchmarks, TabPATE preserves competitive utility while reducing membership inference to near-random success, providing a practical path to private tabular ICL without public data.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 29 of its 33 references that are in the map. First posted on arXiv 30 Jun 2026.

Builds on

29 of 33 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Membership Inference Attacks on Diffusion Models via Quantile Regression8 Dec
  2. 2021Membership Inference Attacks From First Principles7 Dec

Other references · 27

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2016Deep Learning with Differential Privacycited by 2,728
  4. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  5. 2020Label-Only Membership Inference Attackscited by 388
  6. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 348
  7. 2025TabICL: A Tabular Foundation Model for In-Context Learning on Large Datacited by 289
  8. 2022TabLLM: Few-shot Classification of Tabular Data with Large Language Modelscited by 277
  9. 2018Scalable Private Learning with PATEcited by 269
  10. 2017OpenML Benchmarking Suitescited by 178
  11. 2025TabArena: A Living Benchmark for Machine Learning on Tabular Datacited by 176
  12. 2026TabICLv2: A better, faster, scalable, and open tabular foundation modelcited by 144
15 more
  1. 2022The Privacy Onion Effect: Memorization is Relativecited by 136
  2. 2024TabDPT: Scaling Tabular Foundation Models on Real Datacited by 113
  3. 2021Winning the NIST Contest: A scalable and general approach to differentially private synthetic datacited by 102
  4. 2022Truth Serum: Poisoning Machine Learning Models to Reveal Their Secretscited by 101
  5. 2023Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generationcited by 96
  6. 2024Better by Default: Strong Pre-Tuned MLPs and Boosted Trees on Tabular Datacited by 92
  7. 2022AIM: An Adaptive and Iterative Mechanism for Differentially Private Synthetic Datacited by 90
  8. 2023Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Modelscited by 83
  9. 2019Graphical-model based estimation and inference for differential privacycited by 74
  10. 2023Privacy-Preserving In-Context Learning for Large Language Modelscited by 50
  11. 2025CausalPFN: Amortized Causal Effect Estimation via In-Context Learningcited by 49
  12. 2024DP-TabICL: In-Context Learning with Differentially Private Tabular Datacited by 11
  13. 2025Trustworthy AI Must Account for Interactionscited by 5
  14. 2026Causal Foundation Models with Continuous Treatmentscited by 4
  15. 2025Are Foundation Models Useful for Bankruptcy Prediction?cited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Differentially Private Synthetic Data via APIs 4: Tabular Data
  2. 2025Risk In Context: Benchmarking Privacy Leakage of Foundation Models in Synthetic Tabular Data Generation
  3. 2026Privacy Vulnerabilities of Attention Layers in Tabular Foundation Models and Protection of High-Risk Queries
  4. 2026TabKD: Tabular Knowledge Distillation through Interaction Diversity of Learned Feature Bins
  5. 2022An Empirical Study on the Membership Inference Attack against Tabular Data Synthesis Models
  6. 2025Privacy Preserving Diffusion Models for Mixed-Type Tabular Data Generation
  7. 2026SoK: Rethinking Membership Inference in Tabular Machine Learning
  8. 2024HARMONIC: Harnessing LLMs for Tabular Data Synthesis and Privacy Protection