mlsanityData to 7 Oct 2026

Con-ReCall: Detecting Pre-training Data in LLMs via Contrastive Decoding

arXiv 2409.03363 ↗First posted 5 Sep 2024cs.CLcited by 20

Not on the map. Why?

Abstract (arXiv)

The training data in large language models is key to their success, but it also presents privacy and security risks, as it may contain sensitive information. Detecting pre-training data is crucial for mitigating these concerns. Existing methods typically analyze target text in isolation or solely with non-member contexts, overlooking potential insights from simultaneously considering both member and non-member contexts. While previous work suggested that member contexts provide little information due to the minor distributional shift they induce, our analysis reveals that these subtle shifts can be effectively leveraged when contrasted with non-member contexts. In this paper, we propose Con-ReCall, a novel approach that leverages the asymmetric distributional shifts induced by member and non-member contexts through contrastive decoding, amplifying subtle differences to enhance membership inference. Extensive empirical evaluations demonstrate that Con-ReCall achieves state-of-the-art performance on the WikiMIA benchmark and is robust against various text manipulation techniques.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 25 of its 30 references that are in the map. First posted on arXiv 5 Sep 2024.

Builds on

25 of 30 references.

References · 25

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  3. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  5. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  6. 2021Membership Inference Attacks From First Principlescited by 839
  7. 2023Extracting Training Data from Diffusion Modelscited by 723
  8. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  9. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  10. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 339
  11. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  12. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 249
13 more
  1. 2023Proving Test Set Contamination in Black Box Language Modelscited by 227
  2. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attackscited by 168
  3. 2021On the Importance of Difficulty Calibration in Membership Inference Attackscited by 137
  4. 2023Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4cited by 123
  5. 2024Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Modelscited by 102
  6. 2023Low-Cost High-Power Membership Inference Attackscited by 99
  7. 2023Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generationcited by 94
  8. 2023Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibrationcited by 90
  9. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 76
  10. 2024DE-COP: Detecting Copyrighted Content in Language Models Training Datacited by 72
  11. 2023Scalable Membership Inference Attacks via Quantile Regressioncited by 61
  12. 2024ReCaLL: Membership Inference via Relative Conditional Log-Likelihoodscited by 61
  13. 2024Enhancing Contextual Understanding in Large Language Models through Contrastive Decodingcited by 49

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 9; 2026: 9.

20 papers.

Sort:

By area:

  1. 2024Scaling Up Membership Inference: When and How Attacks Succeed on Large Language Modelscited by 21
  2. 2025False Sense of Security: Why Probing-based Malicious Input Detection Fails to Generalizecited by 15
  3. 2025SOFT: Selective Data Obfuscation for Protecting LLM Fine-tuning against Membership Inference Attackscited by 9
  4. 2025Auditing Data Membership in Reinforcement Learning With Verifiable Rewardscited by 6
  5. 2026Window-based Membership Inference Attacks Against Fine-tuned Large Language Modelscited by 5
  6. 2026Membership Inference Attacks Against Fine-tuned Diffusion Language Modelscited by 5
  7. 2025In-Context Probing for Membership Inference in Fine-Tuned Language Modelscited by 3
  8. 2024On the Validity of Traditional Vulnerability Scoring Systems for Adversarial Attacks against LLMscited by 2
  9. 2026Membership Inference on LLMs in the Wildcited by 2
  10. 2026Reading the Finetuning Prior: Verbatim Content Recovery via Contrastive Decoding Diffingcited by 2
  11. 2025On Membership Inference Attacks in Knowledge Distillationcited by 1
  12. 2025AnalogFed: Privacy-Preserving Discovery of Analog Circuits at Scale with Federated Generative AIcited by 0
8 more
  1. 2025Fast-MIA: Efficient and Scalable Membership Inference for LLMscited by 0
  2. 2025EL-MIA: Quantifying Membership Inference Risks of Sensitive Entities in LLMscited by 0
  3. 2025On the Effectiveness of Membership Inference in Targeted Data Extraction from Large Language Modelscited by 0
  4. 2026On the Evidentiary Limits of Membership Inference for Copyright Auditingcited by 0
  5. 2026Detecting Data Contamination in Large Language Modelscited by 0
  6. 2026Revealing Training Data Exposure in Vision Language Large Models via Parameter Gradientscited by 0
  7. 2026Black-Box Membership Inference via Word-Level Probability Estimationcited by 0
  8. 2026Weak Ties, Strong Signals: Efficient Training Data Detection in Diffusion LLMs via Independent Token Samplingcited by 0