mlsanityData to 14 Oct 2026

JUMP: Single-Pass Membership Inference on Fine-Tuned Diffusion Language Models

arXiv 2607.16207 ↗First posted 9 May 2026cs.AI cs.CRcited by 0

On the map: Diffusion models → Discrete and text diffusion

also in 1 more
Abstract (arXiv)

Membership inference attacks (MIAs) test whether a candidate example was used to train a language model. Existing attacks on fine-tuned discrete diffusion language models (dLLMs) often aggregate reconstruction signals across many mask configurations, requiring repeated model evaluations. We propose Joint Uncertainty Guided Mask Probing (JUMP), an efficient MIA that exploits the ability of dLLMs to predict masked tokens in parallel. Using the pre-fine-tuning checkpoint as a reference, JUMP selects low-confidence positions, masks them jointly, and aggregates clipped target-reference reconstruction gaps. This focuses the attack on positions that reveal stronger membership signals from fine-tuning. After mask selection, all selected tokens are evaluated with one scoring query per model. Across six MIMIR domains, JUMP improves mean ROC-AUC over a prior multi-mask attack from 0.819 to 0.902 on LLaDA and from 0.851 to 0.942 on Dream. Including mask selection, it requires only three forward passes per example, compared with 32 for the baseline. We further extend JUMP to the target-only setting by replacing target-reference scoring with relative token preference, which compares the observed token with alternative predictions at the same masked position. Target-Only JUMP achieves mean ROC-AUCs of 0.609 and 0.638 on LLaDA and Dream.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 37 of its 44 references that are in the map. First posted on arXiv 9 May 2026.

Builds on

37 of 44 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Fine-Tuning Masked Diffusion for Provable Self-Correction1 Oct
  2. 2025Dream 7B: Diffusion Large Language Models21 Aug
  3. 2025Large Language Diffusion Models14 Feb
  4. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  5. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  6. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  7. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  8. 2021Autoregressive Diffusion Models5 Oct

Other references · 29

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  3. 2016Deep Learning with Differential Privacycited by 2,728
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  5. 2020Extracting Training Data from Large Language Modelscited by 2,189
  6. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  7. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
  8. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  9. 2021Membership Inference Attacks From First Principlescited by 878
  10. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  11. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  12. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
17 more
  1. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 412
  2. 2021Differentially Private Fine-tuning of Language Modelscited by 400
  3. 2023Detecting Pretraining Data from Large Language Modelscited by 387
  4. 2015MADE: Masked Autoencoder for Distribution Estimationcited by 307
  5. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 285
  6. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 258
  7. 2024Do Membership Inference Attacks Work on Large Language Models?cited by 195
  8. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attackscited by 175
  9. 2021On the Importance of Difficulty Calibration in Membership Inference Attackscited by 147
  10. 2023Privacy Auditing with One (1) Training Runcited by 123
  11. 2022Measuring Forgetting of Memorized Training Examplescited by 118
  12. 2023Low-Cost High-Power Membership Inference Attackscited by 107
  13. 2018Understanding Membership Inferences on Well-Generalized Learning Modelscited by 90
  14. 2013A Deep and Tractable Density Estimatorcited by 68
  15. 2024ReCaLL: Membership Inference via Relative Conditional Log-Likelihoodscited by 64
  16. 2024Context-Aware Membership Inference Attacks against Pre-trained Large Language Modelscited by 22
  17. 2026Membership Inference Attacks Against Fine-tuned Diffusion Language Modelscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026CoDR: Training-Free Confidence-Drift Remasking for Diffusion Language Models
  2. 2026Re-evaluating Confidence Remasking in Masked Diffusion Language Models
  3. 2026Attention-Discounted Adaptive Sampler for Masked Diffusion Language Models
  4. 2026Fork-dLLM: Avoiding the Flexibility Trap in Diffusion Language Models
  5. 2025Beyond Static Cutoffs: One-Shot Dynamic Thresholding for Diffusion Language Models
  6. 2026CaRE Compute-aware Remasking Evaluation Protocol for Masked Diffusion Language Models
  7. 2026LESS Is More: Mutual-Stability Sampling for Diffusion Language Models
  8. 2026Uncertainty Quantification for Large Language Diffusion Models