mlsanityData to 14 Oct 2026

DMRL: Data- and Model-aware Reward Learning for Data Extraction

arXiv 2505.06284 ↗First posted 7 May 2025cs.LG cs.CRcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

Abstract (arXiv)

Large language models (LLMs) are inherently vulnerable to unintended privacy breaches. Consequently, systematic red-teaming research is essential for developing robust defense mechanisms. However, current data extraction methods suffer from several limitations: (1) rely on dataset duplicates (addressable via deduplication), (2) depend on prompt engineering (now countered by detection and defense), and (3) rely on random-search adversarial generation. To address these challenges, we propose DMRL, a Data- and Model-aware Reward Learning approach for data extraction. This technique leverages inverse reinforcement learning to extract sensitive data from LLMs. Our method consists of two main components: (1) constructing an introspective reasoning dataset that captures leakage mindsets to guide model behavior, and (2) training reward models with Group Relative Policy Optimization (GRPO), dynamically tuning optimization based on task difficulty at both the data and model levels. Comprehensive experiments across various LLMs demonstrate that DMRL outperforms all baseline methods in data extraction performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 35 of its 50 references that are in the map. First posted on arXiv 7 May 2025.

Builds on

35 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset10 Jul

Other references · 34

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  5. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  6. 2020Extracting Training Data from Large Language Modelscited by 2,189
  7. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  8. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
  9. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
  10. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 460
  11. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  12. 2021Differentially Private Fine-tuning of Language Modelscited by 400
22 more
  1. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 314
  2. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 285
  3. 2020Supervised Contrastive Learning for Pre-trained Language Model Fine-tuningcited by 276
  4. 2019Neural Legal Judgment Prediction in Englishcited by 209
  5. 2019Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inferencecited by 175
  6. 2024PLeak: Prompt Leaking Attacks against Large Language Model Applicationscited by 127
  7. 2022Measuring Forgetting of Memorized Training Examplescited by 118
  8. 2021Does BERT Pretrained on Clinical Notes Reveal Sensitive Data?cited by 93
  9. 2016Yelp Dataset Challenge: Review Rating Predictioncited by 84
  10. 2023Bag of Tricks for Training Data Extraction from Language Modelscited by 70
  11. 2023Assessing Prompt Injection Risks in 200+ Custom GPTscited by 55
  12. 2022Models of human preference for learning reward functionscited by 54
  13. 2022Maximum-Likelihood Inverse Reinforcement Learning with Finite-Time Guaranteescited by 42
  14. 2024Getting More Juice Out of the SFT Data: Reward Learning from Human Demonstration Improves SFT for LLM Alignmentcited by 32
  15. 2024Reinforcement Learning from Multi-role Debates as Feedback for Bias Mitigation in LLMscited by 22
  16. 2025Inverse Reinforcement Learning with Dynamic Reward Scaling for LLM Alignmentcited by 20
  17. 2024PBI-Attack: Prior-Guided Bimodal Interactive Black-Box Jailbreak Attack for Toxicity Maximizationcited by 17
  18. 2024AGR: Age Group fairness Reward for Bias Mitigation in LLMscited by 12
  19. 2024Gibberish is All You Need for Membership Inference Detection in Contrastive Language-Audio Pretrainingcited by 10
  20. 2024SelfPrompt: Autonomously Evaluating LLM Robustness via Domain-Constrained Knowledge Guidelines and Refined Adversarial Promptscited by 10
  21. 2022Combing for Credentials: Active Pattern Extraction from Smart Replycited by 7
  22. 2025From Demonstrations to Rewards: Alignment Without Explicit Human Preferencescited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Data Security in Large Language Models: Risks, Defense, and Directions
  2. 2025From Poisoned to Aware: Fostering Backdoor Self-Awareness in LLMs
  3. 2025Defending Against Prompt Injection with DataFilter
  4. 2026Reinforcement Unlearning via Group Relative Policy Optimization
  5. 2025Detecting Data Contamination from Reinforcement Learning Post-training for Large Language Models
  6. 2025RL-Finetuned LLMs for Privacy-Preserving Synthetic Rewriting
  7. 2025Improving LLM Reasoning for Vulnerability Detection via Group Relative Policy Optimization
  8. 2024LeakAgent: RL-based Red-teaming Agent for LLM Privacy Leakage