# Self-Consuming Generative Models with Adversarially Curated Data

arXiv: https://arxiv.org/abs/2505.09768 · HTML: https://mlsanity.com/paper/2505.09768/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 42 references that are in the map.
- First posted on arXiv 14 May 2025.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Self-Consuming Generative Models Go MAD](https://mlsanity.com/paper/2307.01850/) (2023)
- [Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data](https://mlsanity.com/paper/2404.01413/) (2024)
- [Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators](https://mlsanity.com/paper/2403.16950/) (2024)
- [On the Stability of Iterative Retraining of Generative Models on their own Data](https://mlsanity.com/paper/2310.00429/) (2023)
- [Label Sanitization against Label Flipping Poisoning Attacks](https://mlsanity.com/paper/1803.00992/) (2018)
- [RMB: Comprehensively Benchmarking Reward Models in LLM Alignment](https://mlsanity.com/paper/2410.09893/) (2024)
- [Data Feedback Loops: Model-driven Amplification of Dataset Biases](https://mlsanity.com/paper/2209.03942/) (2022)
- [Fairness Feedback Loops: Training on Synthetic Data Amplifies Bias](https://mlsanity.com/paper/2403.07857/) (2024)
- [Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferences](https://mlsanity.com/paper/2407.09499/) (2024)
- [Self-Correcting Self-Consuming Loops for Generative Model Training](https://mlsanity.com/paper/2402.07087/) (2024)
- [Best-of-Venom: Attacking RLHF by Injecting Poisoned Preference Data](https://mlsanity.com/paper/2404.05530/) (2024)
- [Self-Improving Diffusion Models with Synthetic Data](https://mlsanity.com/paper/2408.16333/) (2024)
- [Practical Data Poisoning Attack against Next-Item Recommendation](https://mlsanity.com/paper/2004.03728/) (2020)
- [Would Deep Generative Models Amplify Bias in Future Models?](https://mlsanity.com/paper/2404.03242/) (2024)
- [Preference Poisoning Attacks on Reward Model Learning](https://mlsanity.com/paper/2402.01920/) (2024)
- [Forcing Generative Models to Degenerate Ones: The Power of Data Poisoning Attacks](https://mlsanity.com/paper/2312.04748/) (2023)
- [Automating Data Annotation under Strategic Human Agents: Risks and Potential Solutions](https://mlsanity.com/paper/2405.08027/) (2024)
- [What Distributions are Robust to Indiscriminate Poisoning Attacks for Linear Learners?](https://mlsanity.com/paper/2307.01073/) (2023)

## Built on by (most cited first, 10 of 10)

- [Escaping Model Collapse via Synthetic Data Verification: Near-term Improvements and Long-term Convergence](https://mlsanity.com/paper/2510.16657/) (2025)
- [Stabilizing Self-Consuming Diffusion Models with Latent Space Filtering](https://mlsanity.com/paper/2511.12742/) (2025)
- [Observations and Remedies for Large Language Model Bias in Self-Consuming Performative Loop](https://mlsanity.com/paper/2601.05184/) (2026)
- [When and How Human Curation Backfires: Preference Alignment under Multi-Model Self-Consuming Loop](https://mlsanity.com/paper/2605.29267/) (2026)
- [On-device Large Multi-modal Agent for Human Activity Recognition](https://mlsanity.com/paper/2512.19742/) (2025)
- [When Sample Selection Bias Precipitates Model Collapse](https://mlsanity.com/paper/2606.13732/) (2026)
- [Market Games for Generative Models: Equilibria, Welfare, and Strategic Entry](https://mlsanity.com/paper/2602.17787/) (2026)
- [Recursive LLM Degradation in Biomedical Question Answering: A Cross-Generation Study](https://mlsanity.com/paper/2609.34257/) (2026)
- [Stability and Diversity of Networked Self-Consuming Generative Ecosystems](https://mlsanity.com/paper/2610.09409/) (2026)
- [Self-Consuming Generative Models with Co-Evolving Human Preferences](https://mlsanity.com/paper/2610.09415/) (2026)

## Abstract (arXiv)

Recent advances in generative models have made it increasingly difficult to distinguish real data from model-generated synthetic data. Using synthetic data for successive training of future model generations creates "self-consuming loops", which may lead to model collapse or training instability. Furthermore, synthetic data is often subject to human feedback and curated by users based on their preferences. Ferbach et al. (2024) recently showed that when data is curated according to user preferences, the self-consuming retraining loop drives the model to converge toward a distribution that optimizes those preferences. However, in practice, data curation is often noisy or adversarially manipulated. For example, competing platforms may recruit malicious users to adversarially curate data and disrupt rival models. In this paper, we study how generative models evolve under self-consuming retraining loops with noisy and adversarially curated data. We theoretically analyze the impact of such noisy data curation on generative models and identify conditions for the robustness of the retraining process. Building on this analysis, we design attack algorithms for competitive adversarial scenarios, where a platform with a limited budget employs malicious users to misalign a rival's model from actual user preferences. Experiments on both synthetic and real-world datasets demonstrate the effectiveness of the proposed algorithms.