# Improving Generative Model Self-Training with Geometrically Modified Outputs

arXiv: https://arxiv.org/abs/2609.35512 · HTML: https://mlsanity.com/paper/2609.35512/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 36 references that are in the map.
- First posted on arXiv 28 Sep 2026.

## Builds on (most cited first, 29 of 29)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Improved Precision and Recall Metric for Assessing Generative Models](https://mlsanity.com/paper/1904.06991/) (2019)
- [SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers](https://mlsanity.com/paper/2401.08740/) (2024)
- [Mean Flows for One-step Generative Modeling](https://mlsanity.com/paper/2505.13447/) (2025)
- [Scaling Laws for Autoregressive Generative Modeling](https://mlsanity.com/paper/2010.14701/) (2020)
- [Reliable Fidelity and Diversity Metrics for Generative Models](https://mlsanity.com/paper/2002.09797/) (2020)
- [Scaling Data-Constrained Language Models](https://mlsanity.com/paper/2305.16264/) (2023)
- [One Step Diffusion via Shortcut Models](https://mlsanity.com/paper/2410.12557/) (2024)
- [Is synthetic data from generative models ready for image recognition?](https://mlsanity.com/paper/2210.07574/) (2022)
- [Guiding a Diffusion Model with a Bad Version of Itself](https://mlsanity.com/paper/2406.02507/) (2024)
- [Self-Consuming Generative Models Go MAD](https://mlsanity.com/paper/2307.01850/) (2023)
- [Better Diffusion Models Further Improve Adversarial Training](https://mlsanity.com/paper/2302.04638/) (2023)
- [Will we run out of data? Limits of LLM scaling based on human-generated data](https://mlsanity.com/paper/2211.04325/) (2022)
- [Generative Modeling via Drifting](https://mlsanity.com/paper/2602.04770/) (2026)
- [Inductive Moment Matching](https://mlsanity.com/paper/2503.07565/) (2025)
- [Refining Generative Process with Discriminator Guidance in Score-based Diffusion Models](https://mlsanity.com/paper/2211.17091/) (2022)
- [Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generation](https://mlsanity.com/paper/2402.10210/) (2024)
- [Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification](https://mlsanity.com/paper/2406.07515/) (2024)
- [AlphaFlow: Understanding and Improving MeanFlow Models](https://mlsanity.com/paper/2510.20771/) (2025)
- [Representation Fr'echet Loss for Visual Generation](https://mlsanity.com/paper/2604.28190/) (2026)
- [Self-Improving Diffusion Models with Synthetic Data](https://mlsanity.com/paper/2408.16333/) (2024)
- [Direct Discriminative Optimization: Your Likelihood-Based Visual Generative Model is Secretly a GAN Discriminator](https://mlsanity.com/paper/2503.01103/) (2025)
- [Image Generation with a Sphere Encoder](https://mlsanity.com/paper/2602.15030/) (2026)
- [Neon: Negative Extrapolation From Self-Training Improves Image Generation](https://mlsanity.com/paper/2510.03597/) (2025)

## Abstract (arXiv)

Self-training generative models - the continued improvement of a model using its own outputs - is becoming increasingly important as high-quality training data becomes scarce. However, naively finetuning on model-generated samples leads to degradation through model collapse and the model autophagy disorder. Negative-guidance self-training methods turn this degradation into a useful signal, using a model finetuned on its own outputs to guide the original model toward improved generation. Existing methods, however, take the negative signal in standard model outputs as given. We instead ask whether this signal can be explicitly strengthened. We introduce Geometrically Modified Outputs (GMOs), which reweight the singular values of the generator's input-output Jacobian to increase the influence of its leading singular directions. This geometric modification amplifies the mode-seeking behavior and distortions of standard outputs, providing a stronger and more targeted negative signal for self-training. Across a range of one-step generative models, GMOs consistently improve the performance of negative-guidance methods, including Neon and SIMS, compared with using standard model outputs.