mlsanityData to 7 Oct 2026

When T2I Synthetic Data Backfires: Amplified Privacy Risks in Real-Synthetic Mix Training

arXiv 2607.13541 ↗First posted 15 Jul 2026cs.CR cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

To overcome data scarcity and privacy constraints in data collection, it has become standard practice across academia and industry to augment real training data with text-to-image (T2I)-generated synthetic data, a paradigm we term Real-Synthetic Mix-Training (RSMT). While substituting synthetic data for sensitive real samples is widely regarded as a means to mitigate privacy exposure of the substituted data, the risk to the remaining real samples that actively participate in training has remained largely unexamined. This work reveals, for the first time, that RSMT can substantially amplify privacy leakage of these real training samples. We establish a theoretical framework, RSMT Memorization Amplification, proving that incorporating synthetic data displaces real samples toward peripheral regions of the mixed feature space, in turn forcing the model to memorize them more aggressively. Guided by this foundation, we propose RSMixLeak to systematically assess this risk through membership inference attacks (MIAs). RSMixLeak comprises two variants depending on the adversary's capability. The non-adversarial variant audits a benign RSMT pipeline with an honest T2I provider, establishing a lower bound on the leakage induced by the intrinsic gap between real and T2I-generated data. The adversarial variant considers an adversary who controls the T2I model or contributes crafted data to the T2I provider, and deliberately enlarges this distributional gap on a target class via either high-level semantic attribute binding or imperceptible pixel-level coating, further amplifying leakage on real training data while improving downstream model utility. Motivated by these findings, we further propose a lightweight leakage propensity indicator computable from real data alone that reliably identifies high-risk datasets unsuitable for entering RSMT, as a self-assessable mitigation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 72 of its 99 references that are in the map. First posted on arXiv 15 Jul 2026.

Builds on

72 of 99 references.

References · 72

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2022Scalable Diffusion Models with Transformerscited by 5,931
  5. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  7. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  8. 2016Deep Learning with Differential Privacycited by 2,682
  9. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  10. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  11. 2019Contrastive Multiview Codingcited by 1,297
  12. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
38 more
  1. 2017A Closer Look at Memorization in Deep Networkscited by 936
  2. 2017VGGFace2: A dataset for recognising faces across pose and agecited by 911
  3. 2021Membership Inference Attacks From First Principlescited by 839
  4. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  5. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
  6. 2020Dataset Condensation with Gradient Matchingcited by 510
  7. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  8. 2023Towards Universal Fake Image Detectors that Generalize Across Generative Modelscited by 483
  9. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
  10. 2022Dataset Distillation by Matching Training Trajectoriescited by 437
  11. 2023Synthetic Data from Diffusion Models Improves ImageNet Classificationcited by 326
  12. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  13. 2021Enhanced Membership Inference Attacks against Machine Learning Modelscited by 237
  14. 2020When Machine Unlearning Jeopardizes Privacycited by 181
  15. 2022Fake it till you make it: Learning transferable representations from synthetic ImageNet clonescited by 171
  16. 2021Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Pathcited by 156
  17. 2019Privacy Risks of Securing Machine Learning Models against Adversarial Examplescited by 136
  18. 2017Augmented Reality Meets Computer Vision : Efficient Data Generation for Urban Driving Scenescited by 122
  19. 2022Privacy for Free: How does Dataset Condensation Help Privacy?cited by 119
  20. 2021LENS: Localization enhanced by NeRF synthesiscited by 116
  21. 2023Scaling Laws of Synthetic Images for Model Training ... for Nowcited by 107
  22. 2023Low-Cost High-Power Membership Inference Attackscited by 99
  23. 2017PatternNet: A Benchmark Dataset for Performance Evaluation of Remote Sensing Image Retrievalcited by 96
  24. 2022Membership Inference Attacks by Exploiting Loss Trajectorycited by 93
  25. 2022NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fieldscited by 86
  26. 2021Practical Blind Membership Inference Attack via Differential Comparisonscited by 73
  27. 2021EncoderMI: Membership Inference against Pre-trained Encoders in Contrastive Learningcited by 68
  28. 2024Any-Resolution AI-Generated Image Detection by Spectral Learningcited by 56
  29. 2024Watermarking Makes Language Models Radioactivecited by 52
  30. 2023Did the Neurons Read your Book? Document-level Membership Inference for Large Language Modelscited by 51
  31. 2022Membership Inference Attacks and Defenses in Neural Network Pruningcited by 42
  32. 2024Is Synthetic Data all We Need? Benchmarking the Robustness of Models Trained with Synthetic Imagescited by 41
  33. 2024Membership Inference Attacks Against In-Context Learningcited by 38
  34. 2023Real-Fake: Effective Training Data Synthesis Through Distribution Matchingcited by 35
  35. 2023Black-box Membership Inference Attacks against Fine-tuned Diffusion Modelscited by 35
  36. 2025Towards Label-Only Membership Inference Attack against Pre-trained Large Language Modelscited by 34
  37. 2025Membership Inference Attacks Against Vision-Language Modelscited by 26
  38. 2025Riddle Me This! Stealthy Membership Inference for Retrieval-Augmented Generationcited by 26

50 most cited of 72 All 72 (JSON)

Built on by

Not cited yet.