# How To Think About End-To-End Encryption and AI: Training, Processing, Disclosure, and Consent

arXiv: https://arxiv.org/abs/2412.20231 · HTML: https://mlsanity.com/paper/2412.20231/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 135 references that are in the map.
- First posted on arXiv 28 Dec 2024.

## Builds on (most cited first, 12 of 12)

- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)
- [What Does it Mean for a Language Model to Preserve Privacy?](https://mlsanity.com/paper/2202.05520/) (2022)
- [How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy](https://mlsanity.com/paper/2303.00654/) (2023)
- [Dark Patterns after the GDPR: Scraping Consent Pop-ups and Demonstrating their Influence](https://mlsanity.com/paper/2001.02479/) (2020)
- [The Files are in the Computer: On Copyright, Memorization, and Generative AI](https://mlsanity.com/paper/2404.12590/) (2024)
- [Rethinking Privacy in Machine Learning Pipelines from an Information Flow Control Perspective](https://mlsanity.com/paper/2311.15792/) (2023)
- [Software Grand Exposure: SGX Cache Attacks Are Practical](https://mlsanity.com/paper/1702.07521/) (2017)
- [Inferring Fine-grained Control Flow Inside SGX Enclaves with Branch Shadowing](https://mlsanity.com/paper/1611.06952/) (2016)
- [Privately Policing Dark Patterns](https://mlsanity.com/paper/2307.07888/) (2023)

## Abstract (arXiv)

End-to-end encryption (E2EE) has become the gold standard for securing communications, bringing strong confidentiality and privacy guarantees to billions of users worldwide. However, the current push towards widespread integration of artificial intelligence (AI) models, including in E2EE systems, raises some serious security concerns. This work performs a critical examination of the (in)compatibility of AI models and E2EE applications. We explore this on two fronts: (1) the integration of AI "assistants" within E2EE applications, and (2) the use of E2EE data for training AI models. We analyze the potential security implications of each, and identify conflicts with the security guarantees of E2EE. Then, we analyze legal implications of integrating AI models in E2EE applications, given how AI integration can undermine the confidentiality that E2EE promises. Finally, we offer a list of detailed recommendations based on our technical and legal analyses, including: technical design choices that must be prioritized to uphold E2EE security; how service providers must accurately represent E2EE security; and best practices for the default behavior of AI features and for requesting user consent. We hope this paper catalyzes an informed conversation on the tensions that arise between the brisk deployment of AI and the security offered by E2EE, and guides the responsible development of new AI features.