mlsanityData to 7 Oct 2026

Provable Differentially Private Computation of the Cross-Attention Mechanism

arXiv 2407.14717 ↗First posted 20 Jul 2024cs.LG cs.AI cs.CRcited by 14

On the map: Diffusion models → Robustness, privacy and adversarial aspects

Abstract (arXiv)

Cross-attention has emerged as a cornerstone module in modern artificial intelligence, underpinning critical applications such as retrieval-augmented generation (RAG), system prompting, and guided stable diffusion. However, this is a rising concern about securing the privacy of cross-attention, as the underlying key and value matrices frequently encode sensitive data or private user information. In this work, we introduce a novel data structure designed to enforce differential privacy (DP) for cross-attention mechanisms, accompanied by provable theoretical guarantees. Specifically, letting n denote the input sequence length, d the feature dimension, R the maximum magnitude of query and key matrices, Rw the maximum magnitude of the value matrix, and r, s, εs the parameters for polynomial kernel methods, our proposed structure achieves widetildeO(ndr2) space and initialization complexity, with a query time of widetildeO(d r2) per token. Moreover, we demonstrate that our mechanism satisfies (ε, δ)-DP, incurring an additive error of widetildeO((1-εs)-1 n-1 ε-1 R2s Rw r2) and a relative error of 2εs/(1-εs) with respect to the ground truth. Crucially, our framework maintains robustness against adaptive queries, ensuring security even in adversarial settings. To the best of our knowledge, this constitutes the first approach providing provable differential privacy for cross-attention, establishing a foundation for future privacy-preserving algorithms in large generative models (LGMs).

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 62 of its 95 references that are in the map. First posted on arXiv 20 Jul 2024.

Builds on

62 of 95 references.

References · 62

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2022Scalable Diffusion Models with Transformerscited by 5,931
  5. 2022Emergent Abilities of Large Language Modelscited by 2,483
  6. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  7. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  8. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  9. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  10. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  11. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  12. 2021Differentially Private Fine-tuning of Language Modelscited by 392
38 more
  1. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 320
  2. 2014Sketching as a Tool for Numerical Linear Algebracited by 285
  3. 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 150
  4. 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Modelscited by 111
  5. 2023Fast Attention Requires Bounded Entriescited by 102
  6. 2023Dynamic Prompt Learning: Addressing Cross-Attention Leakage for Text-Based Image Editingcited by 76
  7. 2022Differentially Private Language Models for Secure Data Sharingcited by 68
  8. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  9. 2022Mixed Differential Privacy in Computer Visioncited by 53
  10. 2023On the Role of Attention in Prompt-tuningcited by 49
  11. 2023Privacy in Large Language Models: Attacks, Defenses and Future Directionscited by 48
  12. 2022Robustness Implies Privacy in Statistical Estimationcited by 47
  13. 2023How to Capture Higher-order Correlations? Generalizing Matrix Softmax Attention to Kronecker Computationcited by 44
  14. 2024Why Larger Language Models Do In-context Learning Differently?cited by 44
  15. 2023On Sparse Modern Hopfield Modelcited by 41
  16. 2024On Computational Limits of Modern Hopfield Models: A Fine-Grained Complexity Analysiscited by 41
  17. 2020Local Differential Privacy and Its Applications: A Comprehensive Surveycited by 40
  18. 2022Privately Fine-Tuning Large Language Models with Differential Privacycited by 39
  19. 2024Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Modelscited by 38
  20. 2024Outlier-Efficient Hopfield Layers for Large Transformer-Based Modelscited by 38
  21. 2019A New Analysis of Differential Privacy's Generalization Guaranteescited by 36
  22. 2023STanHop: Sparse Tandem Hopfield Model for Memory-Enhanced Time Series Predictioncited by 35
  23. 2022Just Fine-tune Twice: Selective Differential Privacy for Large Language Modelscited by 32
  24. 2024Towards Few-Shot Adaptation of Foundation Models via Multitask Finetuningcited by 31
  25. 2023Split-and-Denoise: Protect large language model inference with local differential privacycited by 30
  26. 2024The Fine-Grained Complexity of Gradient Computation for Training Large Language Modelscited by 30
  27. 2024Training Tensor Attention Efficiently: From Cubic to Almost Linear Timecited by 29
  28. 2023The Trade-off between Universality and Label Efficiency of Representations from Contrastive Learningcited by 27
  29. 2021Tight and Robust Private Mean Estimation with Few Userscited by 26
  30. 2023Differentially Private Attention Computationcited by 25
  31. 2024Privacy Preserving Prompt Engineering: A Surveycited by 24
  32. 2024Unraveling the Smoothness Properties of Diffusion Models: A Gaussian Mixture Perspectivecited by 24
  33. 2024OmniControlNet: Dual-stage Integration for Conditional Image Generationcited by 24
  34. 2023Dolfin: Diffusion Layout Transformers without Autoencodercited by 23
  35. 2024BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Modelcited by 21
  36. 2023Better and Simpler Lower Bounds for Differentially Private Statistical Estimationcited by 16
  37. 2024Nonparametric Modern Hopfield Modelscited by 16
  38. 2022Adaptive and Dynamic Multi-Resolution Hashing for Pairwise Summationscited by 14

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 7; 2025: 7.

14 papers.

Sort:

By area:

  1. 2024Circuit Complexity Bounds for RoPE-based Transformer Architecturecited by 26
  2. 2024Multi-Layer Transformers Gradient Can be Approximated in Almost Linear Timecited by 22
  3. 2024The Computational Limits of State-Space Models and Mamba via the Lens of Circuit Complexitycited by 21
  4. 2024HSR-Enhanced Sparse Attention Accelerationcited by 19
  5. 2024Theoretical Constraints on the Expressive Power of mathsfRoPE-based Tensor Attention Transformerscited by 12
  6. 2025Integrating Large Language Models for Automated Structural Analysiscited by 12
  7. 2025Theoretical Guarantees for High Order Trajectory Refinement in Generative Flowscited by 9
  8. 2025Fast RoPE Attention: Combining the Polynomial Method and Fast Fourier Transformcited by 9
  9. 2024Fast John Ellipsoid Computation with Differential Privacy Optimizationcited by 7
  10. 2025Only Large Weights (And Not Skip Connections) Can Prevent the Perils of Rank Collapsecited by 7
  11. 2025Scaling Law Phenomena Across Regression Paradigms: Multiple and Kernel Approachescited by 4
  12. 2024Inverting the Leverage Score Gradient: An Efficient Approximate Newton Methodcited by 3
2 more
  1. 2025Time and Memory Trade-off of KV-Cache Compression in Tensor Transformer Decodingcited by 2
  2. 2025Theoretical Foundation of Flow-Based Time Series Generation: Provable Approximation, Generalization, and Efficiencycited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2024Privacy-Preserving Retrieval-Augmented Generation with Differential Privacy
  2. 2026Differentially Private Retrieval-Augmented Generation
  3. 2025RAPID: Retrieval Augmented Training of Differentially Private Diffusion Models
  4. 2026An Efficient and Privacy-Preserving Architecture for Cross-Institutional Collaborative RAG
  5. 2025Private-RAG: Answering Multiple Queries with LLMs while Keeping Your Data Private
  6. 2024DP-RDM: Adapting Diffusion Models to Private Domains Without Fine-Tuning
  7. 2025Dual-Priv Pruning : Efficient Differential Private Fine-Tuning in Multimodal Large Language Models
  8. 2025Differentially Private Synthetic Text Generation for Retrieval-Augmented Generation (RAG)