mlsanityData to 14 Oct 2026

Fragments to Facts: Partial-Information Fragment Inference from LLMs

arXiv 2505.13819 ↗First posted 20 May 2025cs.LG cs.CR cs.CYcited by 0

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Large language models (LLMs) can leak sensitive training data through memorization and membership inference attacks. Prior work has primarily focused on strong adversarial assumptions, including attacker access to entire samples or long, ordered prefixes, leaving open the question of how vulnerable LLMs are when adversaries have only partial, unordered sample information. For example, if an attacker knows a patient has "hypertension," under what conditions can they query a model fine-tuned on patient data to learn the patient also has "osteoarthritis?" In this paper, we introduce a more general threat model under this weaker assumption and show that fine-tuned LLMs are susceptible to these fragment-specific extraction attacks. To systematically investigate these attacks, we propose two data-blind methods: (1) a likelihood ratio attack inspired by methods from membership inference, and (2) a novel approach, PRISM, which regularizes the ratio by leveraging an external prior. Using examples from both medical and legal settings, we show that both methods are competitive with a data-aware baseline classifier that assumes access to labeled in-distribution data, underscoring their robustness.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 61 of its 227 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

61 of 227 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Do Membership Inference Attacks Work on Large Language Models?12 Feb
  2. 2023Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration10 Nov
  3. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparison29 May
  4. 2023Analyzing Leakage of Personally Identifiable Information in Language Models1 Feb
  5. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks8 Mar
  6. 2021Membership Inference Attacks From First Principles7 Dec
  7. 2021Membership Inference Attack Susceptibility of Clinical Language Models16 Apr

Other references · 54

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  8. 2023Mistral 7Bcited by 3,164
  9. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  10. 2020Extracting Training Data from Large Language Modelscited by 2,189
  11. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  12. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
38 more
  1. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  2. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  3. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 412
  4. 2021Differentially Private Fine-tuning of Language Modelscited by 400
  5. 2023Detecting Pretraining Data from Large Language Modelscited by 387
  6. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 257
  7. 2023Proving Test Set Contamination in Black Box Language Modelscited by 230
  8. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 186
  9. 2024Stealing Part of a Production Language Modelcited by 159
  10. 2024Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learningcited by 155
  11. 2023Towards Accurate Differential Diagnosis with Large Language Modelscited by 116
  12. 2023Bag of Tricks for Training Data Extraction from Language Modelscited by 70
  13. 2024Large Language Models for Disease Diagnosis: A Scoping Reviewcited by 51
  14. 2022Contrastive Language-Vision AI Models Pretrained on Web-Scraped Multimodal Data Exhibit Sexual Objectification Biascited by 43
  15. 2024MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuningcited by 31
  16. 2021Privacy Regularization: Joint Privacy-Utility Optimization in Language Modelscited by 30
  17. 2024Characterizing LLM Abstention Behavior in Science QA with Context Perturbationscited by 26
  18. 2016Viziometrics: Analyzing Visual Information in the Scientific Literaturecited by 15
  19. 2017Synthetic Data for Social Goodcited by 15
  20. 2023NLP for Maternal Healthcare: Perspectives and Guiding Principles in the Age of LLMscited by 14
  21. 2025Epistemic Alignment: A Mediating Framework for User-LLM Knowledge Deliverycited by 14
  22. 2019FairST: Equitable Spatial and Temporal Demand Prediction for New Mobility Systemscited by 13
  23. 2018A Nutritional Label for Rankingscited by 10
  24. 2024The Impact and Opportunities of Generative AI in Fact-Checkingcited by 10
  25. 2017LaraDB: A Minimalist Kernel for Linear and Relational Algebra Computationcited by 5
  26. 2022Epistemic Parity: Reproducibility as an Evaluation Metric for Differential Privacycited by 5
  27. 2024PathwayBench: Assessing Routability of Pedestrian Pathway Networks Inferred from Multi-City Imagerycited by 4
  28. 2024Dataset Scale and Societal Consistency Mediate Facial Impression Bias in Vision-Language AIcited by 4
  29. 2024Representation Bias of Adolescents in AI: A Bilingual, Bicultural Studycited by 4
  30. 2019Data Management for Causal Algorithmic Fairnesscited by 3
  31. 2024Towards Zero-Shot Annotation of the Built Environment with Vision-Language Models (Vision Paper)cited by 3
  32. 2024The Implications of Open Generative Models in Human-Centered Data Science Work: A Case Study with Fact-Checking Organizationscited by 3
  33. 2024ML-EAT: A Multilevel Embedding Association Test for Interpretable and Transparent Social Sciencecited by 3
  34. 2017EZLearn: Exploiting Organic Supervision in Large-Scale Data Annotationcited by 2
  35. 2018Database-Agnostic Workload Managementcited by 2
  36. 2024Reliable, Routable, and Reproducible: Collection of Pedestrian Pathways at Statewide Scalecited by 2
  37. 2016From NoSQL Accumulo to NewSQL Graphulo: Design and Utility of Graph Algorithms inside a BigTable Databasecited by 1
  38. 2018MobilityMirror: Bias-Adjusted Transportation Datasetscited by 1

50 most cited of 54 All 54 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Exploring Membership Inference Vulnerabilities in Clinical Large Language Models
  2. 2024Context-Aware Membership Inference Attacks against Pre-trained Large Language Models
  3. 2024Pandora's White-Box: Precise Training Data Detection and Extraction in Large Language Models
  4. 2024Order of Magnitude Speedups for LLM Membership Inference
  5. 2024Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models
  6. 2026AttenMIA: LLM Membership Inference Attack through Attention Signals
  7. 2025Paper Summary Attack: Jailbreaking LLMs through LLM Safety Papers
  8. 2025(Token-Level) InfoRMIA: Stronger Membership Inference and Memorization Assessment for LLMs