mlsanityData to 14 Oct 2026

GEP: A GCG-Based method for extracting personally identifiable information from chatbots built on small language models

arXiv 2509.21192 ↗First posted 25 Sep 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Small language models (SLMs) become unprecedentedly appealing due to their approximately equivalent performance compared to large language models (LLMs) in certain fields with less energy and time consumption during training and inference. However, the personally identifiable information (PII) leakage of SLMs for downstream tasks has yet to be explored. In this study, we investigate the PII leakage of the chatbot based on SLM. We first finetune a new chatbot, i.e., ChatBioGPT based on the backbone of BioGPT using medical datasets Alpaca and HealthCareMagic. It shows a matchable performance in BERTscore compared with previous studies of ChatDoctor and ChatGPT. Based on this model, we prove that the previous template-based PII attacking methods cannot effectively extract the PII in the dataset for leakage detection under the SLM condition. We then propose GEP, which is a greedy coordinate gradient-based (GCG) method specifically designed for PII extraction. We conduct experimental studies of GEP and the results show an increment of up to 60× more leakage compared with the previous template-based methods. We further expand the capability of GEP in the case of a more complicated and realistic situation by conducting free-style insertion where the inserted PII in the dataset is in the form of various syntactic expressions instead of fixed templates, and GEP is still able to reveal a PII leakage rate of up to 4.53%.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 43 of its 52 references that are in the map. First posted on arXiv 25 Sep 2025.

Builds on

43 of 52 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  8. 2023A Survey of Large Language Modelscited by 2,576
  9. 2024Qwen2 Technical Reportcited by 2,120
  10. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  11. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  12. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
31 more
  1. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
  2. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  3. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 723
  4. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  5. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  6. 2024OLMo: Accelerating the Science of Language Modelscited by 613
  7. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 538
  8. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  9. 2023ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledgecited by 325
  10. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 301
  11. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 285
  12. 2023Certifying LLM Safety against Adversarial Promptingcited by 266
  13. 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 200
  14. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 195
  15. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 186
  16. 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 158
  17. 2023ProPILE: Probing Privacy Leakage in Large Language Modelscited by 129
  18. 2024A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthinesscited by 127
  19. 2023LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructionscited by 124
  20. 2024On-Device Language Models: A Comprehensive Reviewcited by 104
  21. 2021Does BERT Pretrained on Clinical Notes Reveal Sensitive Data?cited by 93
  22. 2024BioMedLM: A 2.7B Parameter Language Model Trained On Biomedical Textcited by 86
  23. 2024OpenELM: An Efficient Language Model Family with Open Training and Inference Frameworkcited by 75
  24. 2024A Complete Survey on LLM-based AI Chatbotscited by 71
  25. 2023MentaLLaMA: Interpretable Mental Health Analysis on Social Media with Large Language Modelscited by 63
  26. 2021Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domainscited by 36
  27. 2023The Janus Interface: How Fine-Tuning in Large Language Models Amplifies the Privacy Riskscited by 36
  28. 2024PII-Compass: Guiding LLM training data extraction prompts towards the target PII via groundingcited by 24
  29. 2025Should We Still Pretrain Encoders with Masked Language Modeling?cited by 16
  30. 2024Hippocrates: An Open-Source Framework for Advancing Large Language Models in Healthcarecited by 15
  31. 2024H2O-Danube3 Technical Reportcited by 7

Built on by

Not cited yet.