mlsanityData to 7 Oct 2026

Exploring prompts to elicit memorization in masked language model-based named entity recognition

arXiv 2405.03004 ↗First posted 5 May 2024cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Training data memorization in language models impacts model capability (generalization) and safety (privacy risk). This paper focuses on analyzing prompts' impact on detecting the memorization of 6 masked language model-based named entity recognition models. Specifically, we employ a diverse set of 400 automatically generated prompts, and a pairwise dataset where each pair consists of one person's name from the training set and another name out of the set. A prompt completed with a person's name serves as input for getting the model's confidence in predicting this name. Finally, the prompt performance of detecting model memorization is quantified by the percentage of name pairs for which the model has higher confidence for the name from the training set. We show that the performance of different prompts varies by as much as 16 percentage points on the same model, and prompt engineering further increases the gap. Moreover, our experiments demonstrate that prompt performance is model-dependent but does generalize across different name sets. A comprehensive analysis indicates how prompt performance is influenced by prompt properties, contained tokens, and the model's self-attention weights on the prompt.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 30 of its 46 references that are in the map. First posted on arXiv 5 May 2024.

Builds on

30 of 46 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  5. 2016Deep Learning with Differential Privacycited by 2,682
  6. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  7. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  8. 2020Extracting Training Data from Large Language Modelscited by 2,144
  9. 2019How Can We Know What Language Models Know?cited by 953
  10. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  11. 2021Membership Inference Attacks From First Principlescited by 839
  12. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
18 more
  1. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  2. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  3. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
  4. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
  5. 2022Data Contamination: From Memorization to Exploitationcited by 234
  6. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 218
  7. 2019BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Modelcited by 184
  8. 2022Impact of Pretraining Term Frequencies on Few-Shot Reasoningcited by 142
  9. 2018Pathologies of Neural Models Make Interpretations Difficultcited by 140
  10. 2020When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning?cited by 102
  11. 2023GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformercited by 84
  12. 2023Privacy Issues in Large Language Models: A Surveycited by 57
  13. 2023An Overview on Language Models: Recent Developments and Outlookcited by 22
  14. 2024Recent Advances in Named Entity Recognition: A Comprehensive Survey and Comparative Studycited by 19
  15. 2022Parameter-Efficient Tuning Makes a Good Classification Headcited by 15
  16. 2024Take Care of Your Prompt Bias! Investigating and Mitigating Prompt Bias in Factual Knowledge Extractioncited by 13
  17. 2023German BERT Model for Legal Named Entity Recognitioncited by 6
  18. 2022Unintended Memorization and Timing Attacks in Named Entity Recognition Modelscited by 2

Built on by

Not cited yet.