mlsanityData to 14 Oct 2026

How Private is Your Attention? Bridging Privacy with In-Context Learning

arXiv 2504.16000 ↗First posted 22 Apr 2025stat.ML cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

In-context learning (ICL)-the ability of transformer-based models to perform new tasks from examples provided at inference time-has emerged as a hallmark of modern language models. While recent works have investigated the mechanisms underlying ICL, its feasibility under formal privacy constraints remains largely unexplored. In this paper, we propose a differentially private pretraining algorithm for linear attention heads and present the first theoretical analysis of the privacy-accuracy trade-off for ICL in linear regression. Our results characterize the fundamental tension between optimization and privacy-induced noise, formally capturing behaviors observed in private training via iterative methods. Additionally, we show that our method is robust to adversarial perturbations of training prompts, unlike standard ridge regression. All theoretical findings are supported by extensive simulations across diverse settings.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 43 references that are in the map. First posted on arXiv 22 Apr 2025.

Builds on

27 of 43 references.

References · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2016Deep Learning with Differential Privacycited by 2,728
  3. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  4. 2020Extracting Training Data from Large Language Modelscited by 2,189
  5. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  6. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
  7. 2021Membership Inference Attacks From First Principlescited by 878
  8. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 715
  9. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
  10. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 684
  11. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 412
  12. 2021Differentially Private Fine-tuning of Language Modelscited by 400
15 more
  1. 2023Trained Transformers Learn Linear Models In-Contextcited by 337
  2. 2023Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selectioncited by 298
  3. 2021Large-Scale Differentially Private BERTcited by 118
  4. 2022General-Purpose In-Context Learning by Meta-Learning Transformerscited by 105
  5. 2018Privacy Amplification by Iterationcited by 100
  6. 2022Differentially Private Decoding in Large Language Modelscited by 34
  7. 2023In-Context Learning for Attention Scheme: from Single Softmax Regression to Multiple Softmax Regression via a Tensor Trickcited by 32
  8. 2020A Primer on Private Statisticscited by 30
  9. 2024Linear Transformers are Versatile In-Context Learnerscited by 28
  10. 2024Exploring the Frontiers of LLMs in Psychological Applications: A Comprehensive Reviewcited by 17
  11. 2024Differential Privacy Mechanisms in Neural Tangent Kernel Regressioncited by 14
  12. 2016Detecting Communities under Differential Privacycited by 10
  13. 2024Pre-training Differentially Private Models with Limited Public Datacited by 8
  14. 2024Transformers Handle Endogeneity in In-Context Linear Regressioncited by 8
  15. 2024Understanding In-Context Learning of Linear Models in Transformers Through an Adversarial Lenscited by 8

Built on by

Not cited yet.