# Assessing Privacy Risks in Language Models: A Case Study on Summarization Tasks

arXiv: https://arxiv.org/abs/2310.13291 · HTML: https://mlsanity.com/paper/2310.13291/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 20 Oct 2023.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Deduplicating Training Data Makes Language Models Better](https://mlsanity.com/paper/2107.06499/) (2021)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization](https://mlsanity.com/paper/1911.12237/) (2019)
- [Large Language Models Can Be Strong Differentially Private Learners](https://mlsanity.com/paper/2110.05679/) (2021)
- [Label-Only Membership Inference Attacks](https://mlsanity.com/paper/2007.14321/) (2020)
- [Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks](https://mlsanity.com/paper/2203.03929/) (2022)
- [Privacy Risks of Securing Machine Learning Models against Adversarial Examples](https://mlsanity.com/paper/1905.10291/) (2019)
- [Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports](https://mlsanity.com/paper/1911.02541/) (2019)
- [Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?](https://mlsanity.com/paper/1904.05506/) (2019)
- [A Boundary Tilting Persepective on the Phenomenon of Adversarial Examples](https://mlsanity.com/paper/1608.07690/) (2016)
- [THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryption](https://mlsanity.com/paper/2206.00216/) (2022)
- [Membership Inference Attack Susceptibility of Clinical Language Models](https://mlsanity.com/paper/2104.08305/) (2021)
- [A New Defense Against Adversarial Images: Turning a Weakness into a Strength](https://mlsanity.com/paper/1910.07629/) (2019)
- [Membership Inference on Word Embedding and Beyond](https://mlsanity.com/paper/2106.11384/) (2021)
- [Sequence Length is a Domain: Length-based Overfitting in Transformer Models](https://mlsanity.com/paper/2109.07276/) (2021)

## Built on by (most cited first, 8 of 8)

- [Towards Label-Only Membership Inference Attack against Pre-trained Large Language Models](https://mlsanity.com/paper/2502.18943/) (2025)
- [Measuring memorization in RLHF for code completion](https://mlsanity.com/paper/2406.11715/) (2024)
- [Synthetic Data Privacy Metrics](https://mlsanity.com/paper/2501.03941/) (2025)
- [Automated Privacy Information Annotation in Large Language Model Interactions](https://mlsanity.com/paper/2505.20910/) (2025)
- [Uncovering Latent Memories: Assessing Data Leakage and Memorization Patterns in Frontier AI Models](https://mlsanity.com/paper/2406.14549/) (2024)
- [How Private are Language Models in Abstractive Summarization?](https://mlsanity.com/paper/2412.12040/) (2024)
- [AI Security in the Foundation Model Era: A Comprehensive Survey from a Unified Perspective](https://mlsanity.com/paper/2603.24857/) (2026)
- [Have You Ever Seen Them? Entity-level Membership Inference through Interrogating Large Language Models](https://mlsanity.com/paper/2606.23030/) (2026)

## Abstract (arXiv)

Large language models have revolutionized the field of NLP by achieving state-of-the-art performance on various tasks. However, there is a concern that these models may disclose information in the training data. In this study, we focus on the summarization task and investigate the membership inference (MI) attack: given a sample and black-box access to a model's API, it is possible to determine if the sample was part of the training data. We exploit text similarity and the model's resistance to document modifications as potential MI signals and evaluate their effectiveness on widely used datasets. Our results demonstrate that summarization models are at risk of exposing data membership, even in cases where the reference summary is not available. Furthermore, we discuss several safeguards for training summarization models to protect against MI attacks and discuss the inherent trade-off between privacy and utility.