mlsanityData to 7 Oct 2026

Stop Tracking Me! Proactive Defense Against Attribute Inference Attack in LLMs

arXiv 2602.11528 ↗First posted 12 Feb 2026cs.CR cs.AI cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Recent studies have shown that large language models (LLMs) can infer private user attributes (e.g., age, location, gender) from user-generated text shared online, enabling rapid and large-scale privacy breaches. Existing anonymization-based defenses are coarse-grained, lacking word-level precision in anonymizing privacy-leaking elements. Moreover, they are inherently limited as altering user text to hide sensitive cues still allows attribute inference to occur through models' reasoning capabilities. To address these limitations, we propose a unified defense framework that combines fine-grained anonymization (TRACE) with inference-preventing optimization (RPS). TRACE leverages attention mechanisms and inference chain generation to identify and anonymize privacy-leaking textual elements, while RPS employs a lightweight two-stage optimization strategy to induce model rejection behaviors, thereby preventing attribute inference. Evaluations across diverse LLMs show that TRACE-RPS reduces attribute inference accuracy from around 50% to below 5% on open-source models. In addition, our approach offers strong cross-model generalization, prompt-variation robustness, and utility-privacy tradeoffs. Our code is available at https://github.com/Jasper-Yan/TRACE-RPS.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 39 of its 48 references that are in the map. First posted on arXiv 12 Feb 2026.

Builds on

39 of 48 references.

References · 39

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2024Qwen2.5 Technical Reportcited by 4,713
  6. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  8. 2020Extracting Training Data from Large Language Modelscited by 2,144
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  10. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  11. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  12. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
27 more
  1. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  2. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  3. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  4. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  5. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
  6. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 202
  7. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 194
  8. 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 185
  9. 2023Beyond Memorization: Violating Privacy Via Inference with Large Language Modelscited by 167
  10. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 162
  11. 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 150
  12. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attackscited by 150
  13. 2023Open Sesame! Universal Black Box Jailbreaking of Large Language Modelscited by 128
  14. 2023ProPILE: Probing Privacy Leakage in Large Language Modelscited by 127
  15. 2023AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Modelscited by 102
  16. 2024Improved Techniques for Optimization-Based Jailbreaking on Large Language Modelscited by 98
  17. 2024Attention Heads of Large Language Models: A Surveycited by 65
  18. 2024Query-Based Adversarial Prompt Generationcited by 55
  19. 2023Reducing Privacy Risks in Online Self-Disclosures with Language Modelscited by 47
  20. 2024PAL: Proxy-Guided Black-Box Attack on Large Language Modelscited by 46
  21. 2024Identifying Semantic Induction Heads to Understand In-Context Learningcited by 46
  22. 2024A Synthetic Dataset for Personal Attribute Inferencecited by 35
  23. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 22
  24. 2024IncogniText: Privacy-enhancing Conditional Text Anonymization via LLM-based Private Attribute Randomizationcited by 21
  25. 2025Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models?cited by 9
  26. 2024Data Defenses Against Large Language Modelscited by 2
  27. 2025Deploying Privacy Guardrails for LLMs: A Comparative Analysis of Real-World Applicationscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Making MLLMs Blind: Adversarial Smuggling Attacks in MLLM Content Moderationcited by 3
  2. 2026Mitigating the Safety-utility Trade-off in LLM Alignment via Adaptive Safe Context Learningcited by 2
  3. 2026GraphProfiler: Source-Linked Sensitive Attribute Inference via Personal Knowledge Graphscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026On the Privacy of LLMs: An Ablation Study
  2. 2025Ensemble Privacy Defense for Knowledge-Intensive LLMs against Membership Inference Attacks
  3. 2026GRASP: Reinforcing Language Model Anonymizers with Group Relative Policy Optimization
  4. 2025Controlling What You Share: Assessing Language Model Adherence to Privacy Preferences
  5. 2024Tracing Privacy Leakage of Language Models to Training Data via Adjusted Influence Functions
  6. 2025The Canary's Echo: Auditing Privacy Risks of LLM-Generated Synthetic Text
  7. 2026Safer Reasoning Traces: Measuring and Mitigating Chain-of-Thought Leakage in LLMs
  8. 2025Membership Inference Attacks on Large-Scale Models: A Survey