Reducing Privacy Risks in Online Self-Disclosures with Language Models
Not on the map. Why?
Abstract (arXiv)
Self-disclosure, while being common and rewarding in social media interaction, also poses privacy risks. In this paper, we take the initiative to protect the user-side privacy associated with online self-disclosure through detection and abstraction. We develop a taxonomy of 19 self-disclosure categories and curate a large corpus consisting of 4.8K annotated disclosure spans. We then fine-tune a language model for detection, achieving over 65% partial span F1. We further conduct an HCI user study, with 82% of participants viewing the model positively, highlighting its real-world applicability. Motivated by the user feedback, we introduce the task of self-disclosure abstraction, which is rephrasing disclosures into less specific terms while preserving their utility, e.g., "Im 16F" to "I'm a teenage girl". We explore various fine-tuning strategies, and our best model can generate diverse abstractions that moderately reduce privacy risks while maintaining high utility according to human evaluation. To help users in deciding which disclosures to abstract, we present a task of rating their importance for context understanding. Our fine-tuned model achieves 80% accuracy, on-par with GPT-3.5. Given safety and privacy considerations, we will only release our corpus and models to researcher who agree to the ethical guidelines outlined in Ethics Statement.
Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). Builds on 40 of its 142 references that are in the map. First posted on arXiv 16 Nov 2023.
Builds on
40 of 142 references.
References · 40
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
- 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
- 2020Extracting Training Data from Large Language Modelscited by 2,144
- 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
- 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
- 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
- 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
- 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
28 more
- 2016FastText.zip: Compressing text classification modelscited by 331
- 2020LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attentioncited by 326
- 2021All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Textcited by 287
- 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
- 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 218
- 2022CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Taskcited by 205
- 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 195
- 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 150
- 2022The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymizationcited by 94
- 2021Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Textcited by 89
- 2016De-identification of Patient Notes with Recurrent Neural Networkscited by 88
- 2017Depression and Self-Harm Risk Assessment in Online Forumscited by 81
- 2022LENS: A Learnable Evaluation Metric for Text Simplificationcited by 54
- 2023Can Language Models be Instructed to Protect Personal Information?cited by 46
- 2023Training Data Extraction From Pre-trained Language Models: A Surveycited by 43
- 2023Identifying and Mitigating Privacy Risks Stemming from Language Models: A Surveycited by 30
- 2017Multi-Task Learning for Mental Health using Social Media Textcited by 25
- 2023Dancing Between Success and Failure: Edit-level Simplification Evaluation using SALSAcited by 15
- 2020PHICON: Improving Generalization of Clinical Text De-identification Models via Data Augmentationcited by 14
- 2021A Privacy-Preserving Approach to Extraction of Personal Information through Automatic Annotation and Federated Learningcited by 12
- 2022Automatic Identification and Classification of Bragging in Social Mediacited by 12
- 2023Does fine-tuning GPT-3 with the OpenAI API leak personally-identifiable information?cited by 9
- 2022Multilingual Detection of Personal Employment Status on Twittercited by 8
- 2022Improving Large-scale Paraphrase Acquisition and Generationcited by 8
- 2022Behind the Mask: Demographic bias in name detection for PII maskingcited by 7
- 2021MultiTalk: A Highly-Branching Dialog Testbed for Diverse Conversationscited by 6
- 2023Crowdsourcing on Sensitive Data with Privacy-Preserving Text Rewritingcited by 3
- 2020Privacy in Crisis: A study of self-disclosure during the Coronavirus pandemiccited by 2
Built on by
Citing papers per year: 2024: 11; 2025: 20; 2026: 16.
47 papers.
Sort:
By area:
- 2024Trust No Bot: Discovering Personal Disclosures in Human-LLM Conversations in the Wildcited by 43
- 2025The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating Itcited by 42
- 2024A Synthetic Dataset for Personal Attribute Inferencecited by 35
- 2025Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agentscited by 26
- 2024Large Language Models are Advanced Anonymizerscited by 22
- 2024IncogniText: Privacy-enhancing Conditional Text Anonymization via LLM-based Private Attribute Randomizationcited by 21
- 2024Granular Privacy Control for Geolocation with Vision Language Modelscited by 19
- 2024Rescriber: Smaller-LLM-Powered User-Led Data Minimization for LLM-Based Chatbotscited by 16
- 2025A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakagecited by 15
- 2025Position: Privacy Is Not Just Memorization!cited by 15
- 2024The Fire Thief Is Also the Keeper: Balancing Usability and Privacy in Promptscited by 13
- 2025PIG: Privacy Jailbreak Attack on LLMs via Gradient-based Iterative In-Context Optimizationcited by 10
35 more
- 2024Robust Utility-Preserving Text Anonymization Based on Large Language Modelscited by 9
- 2024NAP^2: A Benchmark for Naturalness and Privacy-Preserving Text Rewriting by Learning from Humancited by 8
- 2025Operationalizing Data Minimization for Privacy-Preserving LLM Promptingcited by 8
- 2025Automated Privacy Information Annotation in Large Language Model Interactionscited by 6
- 2025PPMI: Privacy-Preserving LLM Interaction with Socratic Chain-of-Thought Reasoning and Homomorphically Encrypted Vector Databasescited by 6
- 2025Cape: Context-Aware Prompt Perturbation Mechanism with Differential Privacycited by 5
- 2025How do we measure privacy in text? A survey of text anonymization metricscited by 5
- 2024Measuring, Modeling, and Helping People Account for Privacy Risks in Online Self-Disclosures with AIcited by 4
- 2025Can Large Language Models Really Recognize Your Name?cited by 4
- 2025Self-Refining Language Model Anonymizers via Adversarial Distillationcited by 4
- 2025Protecting Vulnerable Voices: Synthetic Dataset Generation for Self-Disclosure Detectioncited by 3
- 2025Zero-Shot Privacy-Aware Text Rewriting via Iterative Tree Searchcited by 3
- 2025Not My Agent, Not My Boundary? Elicitation of Personal Privacy Boundaries in AI-Delegated Information Sharingcited by 3
- 2026Privasis: Synthesizing the Largest "Public" Private Dataset from Scratchcited by 3
- 2026CAPID: Context-Aware PII Detection for Question-Answering Systemscited by 3
- 2026Stop Tracking Me! Proactive Defense Against Attribute Inference Attack in LLMscited by 3
- 2024Model Lakescited by 2
- 2025Learning Obfuscations Of LLM Embedding Sequences: Stained Glass Transformcited by 2
- 2025GAMA: A General Anonymizing Multi-Agent System for Privacy Preservation Enhanced by Domain Rules and Disproof Mechanismcited by 2
- 2025Beyond PII: How Users Attempt to Estimate and Mitigate Implicit LLM Inferencecited by 2
- 2026Privacy in Human-AI Romantic Relationships: Concerns, Boundaries, and Agencycited by 2
- 2026Private Seeds, Public LLMs: Realistic and Privacy-Preserving Synthetic Data Generationcited by 2
- 2026Beyond Direct Identifiers: Probabilistic Privacy Risk Estimation for Privacy-Conscious LLM Query Delegationcited by 2
- 2025Semantically-Aware LLM Agent to Enhance Privacy in Conversational AI Servicescited by 1
- 2026Adaptive Text Anonymization: Learning Privacy-Utility Trade-offs via Prompt Optimizationcited by 1
- 2026GroupGPT: A Token-efficient and Privacy-preserving Agentic Framework for Multi-User Chat Assistantcited by 1
- 2026Selection-Invariant Communication Compilers for Privacy-Aware Multi-Agent LLM Workflowscited by 1
- 2025Text Production and Comprehension by Human and Artificial Intelligence: Interdisciplinary Workshop Reportcited by 0
- 2026BodhiPromptShield: Pre-Inference Prompt Mediation for Surface-Form Privacy Propagation in LLM Agent Pipelinescited by 0
- 2026TikStance: A Multimodal and Hierarchical Dataset for Multi-target Stance Analysis in TikTok Political Conversationscited by 0
- 2026GRASP: Reinforcing Language Model Anonymizers with Group Relative Policy Optimizationcited by 0
- 2026CIDER: A Dataset of Contextual Disclosure Boundaries for Privacy Preference Alignmentcited by 0
- 2026Aggregation-Aware Synthetic Text Generation Against Authorship Re-Identificationcited by 0
- 2026Disclosure-Gated User Simulation for Companion-Agent Evaluationcited by 0
- 2026Conformal Privacy Auditing: Calibrated Re-identification Attacks with Statistical Guaranteescited by 0
- 2026Conformal Privacy Auditing: Calibrated Re-identification Attacks with Statistical Guarantees18 Sep 2026
- 2026Disclosure-Gated User Simulation for Companion-Agent Evaluation1 Sep 2026
- 2026Aggregation-Aware Synthetic Text Generation Against Authorship Re-Identification23 Aug 2026
- 2026CIDER: A Dataset of Contextual Disclosure Boundaries for Privacy Preference Alignment10 Aug 2026
- 2026Beyond Direct Identifiers: Probabilistic Privacy Risk Estimation for Privacy-Conscious LLM Query Delegation10 Aug 2026
- 2026GRASP: Reinforcing Language Model Anonymizers with Group Relative Policy Optimization6 Aug 2026
- 2026Selection-Invariant Communication Compilers for Privacy-Aware Multi-Agent LLM Workflows4 Aug 2026
- 2026TikStance: A Multimodal and Hierarchical Dataset for Multi-target Stance Analysis in TikTok Political Conversations16 Jul 2026
- 2026Private Seeds, Public LLMs: Realistic and Privacy-Preserving Synthetic Data Generation8 Apr 2026
- 2026BodhiPromptShield: Pre-Inference Prompt Mediation for Surface-Form Privacy Propagation in LLM Agent Pipelines7 Apr 2026
- 2026GroupGPT: A Token-efficient and Privacy-preserving Agentic Framework for Multi-User Chat Assistant1 Mar 2026
- 2026Adaptive Text Anonymization: Learning Privacy-Utility Trade-offs via Prompt Optimization24 Feb 2026
35 more
- 2026Stop Tracking Me! Proactive Defense Against Attribute Inference Attack in LLMs12 Feb 2026
- 2026CAPID: Context-Aware PII Detection for Question-Answering Systems10 Feb 2026
- 2026Privasis: Synthesizing the Largest "Public" Private Dataset from Scratch3 Feb 2026
- 2026Privacy in Human-AI Romantic Relationships: Concerns, Boundaries, and Agency23 Jan 2026
- 2025How do we measure privacy in text? A survey of text anonymization metrics30 Nov 2025
- 2025Semantically-Aware LLM Agent to Enhance Privacy in Conversational AI Services30 Oct 2025
- 2025Operationalizing Data Minimization for Privacy-Preserving LLM Prompting4 Oct 2025
- 2025Position: Privacy Is Not Just Memorization!2 Oct 2025
- 2025Not My Agent, Not My Boundary? Elicitation of Personal Privacy Boundaries in AI-Delegated Information Sharing26 Sep 2025
- 2025Zero-Shot Privacy-Aware Text Rewriting via Iterative Tree Search25 Sep 2025
- 2025Beyond PII: How Users Attempt to Estimate and Mitigate Implicit LLM Inference15 Sep 2025
- 2025GAMA: A General Anonymizing Multi-Agent System for Privacy Preservation Enhanced by Domain Rules and Disproof Mechanism12 Sep 2025
- 2025Protecting Vulnerable Voices: Synthetic Dataset Generation for Self-Disclosure Detection24 Jul 2025
- 2025Text Production and Comprehension by Human and Artificial Intelligence: Interdisciplinary Workshop Report28 Jun 2025
- 2025PPMI: Privacy-Preserving LLM Interaction with Socratic Chain-of-Thought Reasoning and Homomorphically Encrypted Vector Databases19 Jun 2025
- 2025Learning Obfuscations Of LLM Embedding Sequences: Stained Glass Transform11 Jun 2025
- 2025Self-Refining Language Model Anonymizers via Adversarial Distillation2 Jun 2025
- 2025The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating It30 May 2025
- 2025Automated Privacy Information Annotation in Large Language Model Interactions27 May 2025
- 2025Can Large Language Models Really Recognize Your Name?20 May 2025
- 2025PIG: Privacy Jailbreak Attack on LLMs via Gradient-based Iterative In-Context Optimization15 May 2025
- 2025Cape: Context-Aware Prompt Perturbation Mechanism with Differential Privacy9 May 2025
- 2025A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakage28 Apr 2025
- 2025Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agents22 Feb 2025
- 2024Measuring, Modeling, and Helping People Account for Privacy Risks in Online Self-Disclosures with AI19 Dec 2024
- 2024Rescriber: Smaller-LLM-Powered User-Led Data Minimization for LLM-Based Chatbots10 Oct 2024
- 2024Robust Utility-Preserving Text Anonymization Based on Large Language Models16 Jul 2024
- 2024Trust No Bot: Discovering Personal Disclosures in Human-LLM Conversations in the Wild16 Jul 2024
- 2024Granular Privacy Control for Geolocation with Vision Language Models6 Jul 2024
- 2024IncogniText: Privacy-enhancing Conditional Text Anonymization via LLM-based Private Attribute Randomization3 Jul 2024
- 2024The Fire Thief Is Also the Keeper: Balancing Usability and Privacy in Prompts20 Jun 2024
- 2024A Synthetic Dataset for Personal Attribute Inference11 Jun 2024
- 2024NAP^2: A Benchmark for Naturalness and Privacy-Preserving Text Rewriting by Learning from Human6 Jun 2024
- 2024Model Lakes4 Mar 2024
- 2024Large Language Models are Advanced Anonymizers21 Feb 2024
- 2025Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agentscited by 26
- 2025GAMA: A General Anonymizing Multi-Agent System for Privacy Preservation Enhanced by Domain Rules and Disproof Mechanismcited by 2
- 2025Semantically-Aware LLM Agent to Enhance Privacy in Conversational AI Servicescited by 1
- 2026GroupGPT: A Token-efficient and Privacy-preserving Agentic Framework for Multi-User Chat Assistantcited by 1
- 2026Selection-Invariant Communication Compilers for Privacy-Aware Multi-Agent LLM Workflowscited by 1
- 2026BodhiPromptShield: Pre-Inference Prompt Mediation for Surface-Form Privacy Propagation in LLM Agent Pipelinescited by 0
- 2026Disclosure-Gated User Simulation for Companion-Agent Evaluationcited by 0
- 2026Disclosure-Gated User Simulation for Companion-Agent Evaluation1 Sep 2026
- 2026Selection-Invariant Communication Compilers for Privacy-Aware Multi-Agent LLM Workflows4 Aug 2026
- 2026BodhiPromptShield: Pre-Inference Prompt Mediation for Surface-Form Privacy Propagation in LLM Agent Pipelines7 Apr 2026
- 2026GroupGPT: A Token-efficient and Privacy-preserving Agentic Framework for Multi-User Chat Assistant1 Mar 2026
- 2025Semantically-Aware LLM Agent to Enhance Privacy in Conversational AI Services30 Oct 2025
- 2025GAMA: A General Anonymizing Multi-Agent System for Privacy Preservation Enhanced by Domain Rules and Disproof Mechanism12 Sep 2025
- 2025Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agents22 Feb 2025
- 2024Trust No Bot: Discovering Personal Disclosures in Human-LLM Conversations in the Wildcited by 43
- 2024Large Language Models are Advanced Anonymizerscited by 22
- 2024IncogniText: Privacy-enhancing Conditional Text Anonymization via LLM-based Private Attribute Randomizationcited by 21
- 2024Rescriber: Smaller-LLM-Powered User-Led Data Minimization for LLM-Based Chatbotscited by 16
- 2025A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakagecited by 15
- 2025Position: Privacy Is Not Just Memorization!cited by 15
- 2024The Fire Thief Is Also the Keeper: Balancing Usability and Privacy in Promptscited by 13
- 2024Robust Utility-Preserving Text Anonymization Based on Large Language Modelscited by 9
- 2024NAP^2: A Benchmark for Naturalness and Privacy-Preserving Text Rewriting by Learning from Humancited by 8
- 2025Operationalizing Data Minimization for Privacy-Preserving LLM Promptingcited by 8
- 2025Automated Privacy Information Annotation in Large Language Model Interactionscited by 6
- 2025Cape: Context-Aware Prompt Perturbation Mechanism with Differential Privacycited by 5
12 most cited of 31 All 31 (JSON)
- 2026Conformal Privacy Auditing: Calibrated Re-identification Attacks with Statistical Guarantees18 Sep 2026
- 2026Aggregation-Aware Synthetic Text Generation Against Authorship Re-Identification23 Aug 2026
- 2026Beyond Direct Identifiers: Probabilistic Privacy Risk Estimation for Privacy-Conscious LLM Query Delegation10 Aug 2026
- 2026TikStance: A Multimodal and Hierarchical Dataset for Multi-target Stance Analysis in TikTok Political Conversations16 Jul 2026
- 2026Adaptive Text Anonymization: Learning Privacy-Utility Trade-offs via Prompt Optimization24 Feb 2026
- 2026Stop Tracking Me! Proactive Defense Against Attribute Inference Attack in LLMs12 Feb 2026
- 2026CAPID: Context-Aware PII Detection for Question-Answering Systems10 Feb 2026
- 2026Privasis: Synthesizing the Largest "Public" Private Dataset from Scratch3 Feb 2026
- 2026Privacy in Human-AI Romantic Relationships: Concerns, Boundaries, and Agency23 Jan 2026
- 2025How do we measure privacy in text? A survey of text anonymization metrics30 Nov 2025
- 2025Operationalizing Data Minimization for Privacy-Preserving LLM Prompting4 Oct 2025
- 2025Position: Privacy Is Not Just Memorization!2 Oct 2025
12 newest of 31 All 31 (JSON)