mlsanityData to 14 Oct 2026

You Don't Know My Favorite Color: Preventing Dialogue Representations from Revealing Speakers' Private Personas

arXiv 2205.10228 ↗First posted 26 Apr 2022cs.CL cs.CR cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Social chatbots, also known as chit-chat chatbots, evolve rapidly with large pretrained language models. Despite the huge progress, privacy concerns have arisen recently: training data of large language models can be extracted via model inversion attacks. On the other hand, the datasets used for training chatbots contain many private conversations between two individuals. In this work, we further investigate the privacy leakage of the hidden states of chatbots trained by language modeling which has not been well studied yet. We show that speakers' personas can be inferred through a simple neural network with high accuracy. To this end, we propose effective defense objectives to protect persona leakage from hidden states. We conduct extensive experiments to demonstrate that our proposed defense objectives can greatly reduce the attack accuracy from 37.6% to 0.5%. Meanwhile, the proposed objectives preserve language models' powerful generation ability.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 26 of its 36 references that are in the map. First posted on arXiv 26 Apr 2022.

Builds on

26 of 36 references.

References · 26

  1. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  2. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  3. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  4. 2020Extracting Training Data from Large Language Modelscited by 2,189
  5. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,077
  6. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,043
  7. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  8. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 828
  9. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 593
  10. 2019TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agentscited by 188
  11. 2018Adversarial Removal of Demographic Attributes from Text Datacited by 170
  12. 2018Dialogue Natural Language Inferencecited by 135
14 more
  1. 2021Does BERT Pretrained on Clinical Notes Reveal Sensitive Data?cited by 93
  2. 2019Analyzing Information Leakage of Updates to Natural Language Modelscited by 90
  3. 2018Learning Controllable Fair Representationscited by 83
  4. 2020DialogXL: All-in-One XLNet for Multi-Party Conversation Emotion Recognitioncited by 79
  5. 2019Overlearning Reveals Sensitive Attributescited by 76
  6. 2018Towards Robust and Privacy-preserving Text Representationscited by 75
  7. 2018Privacy-preserving Neural Representations of Textcited by 54
  8. 2021Privacy Regularization: Joint Privacy-Utility Optimization in Language Modelscited by 30
  9. 2021OTTers: One-turn Topic Transitions for Open-Domain Dialoguecited by 29
  10. 2019Getting To Know You: User Attribute Extraction from Dialoguescited by 19
  11. 2021Honest-but-Curious Nets: Sensitive Attributes of Private Inputs Can Be Secretly Coded into the Classifiers' Outputscited by 18
  12. 2020TIPRDC: Task-Independent Privacy-Respecting Data Crowdsourcing Framework for Deep Learning with Anonymized Intermediate Representationscited by 17
  13. 2019Listening between the Lines: Learning Personal Attributes from Conversationscited by 16
  14. 2021Federated Deep Learning with Bayesian Privacycited by 13

Built on by

Cited by year

Citing papers per year: 2023: 6; 2024: 5.

11 papers.

  1. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 387
  2. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 333
  3. 2023A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospectscited by 66
  4. 2023Privacy in Large Language Models: Attacks, Defenses and Future Directionscited by 51
  5. 2024PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restorationcited by 28
  6. 2024Recent Trends in Personalized Dialogue Generation: A Review of Datasets, Methodologies, and Evaluationscited by 20
  7. 2024Privacy Checklist: Privacy Violation Detection Grounding on Contextual Integrity Theorycited by 18
  8. 2023Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvementscited by 14
  9. 2024No Free Lunch Theorem for Privacy-Preserving LLM Inferencecited by 13
  10. 2023Through the Lens of Core Competency: Survey on Evaluation of Large Language Modelscited by 12
  11. 2024A Survey on Responsible Generative AI: What to Generate and What Notcited by 8