# Collapse of Self-trained Language Models

arXiv: https://arxiv.org/abs/2404.02305 · HTML: https://mlsanity.com/paper/2404.02305/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Self-rewarding alignment and self-training, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 3 of its 8 references that are in the map.
- First posted on arXiv 2 Apr 2024.

## Placed in

- Synthetic data and self-improvement for LLMs / [Self-rewarding alignment and self-training](https://mlsanity.com/synth/self-rewarding-alignment-and-self-training/)

## Builds on (most cited first, 3 of 3)

- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Dynamic Evaluation of Neural Sequence Models](https://mlsanity.com/paper/1709.07432/) (2017)
- [Gradient Amplification: An efficient way to train deep neural networks](https://mlsanity.com/paper/2006.10560/) (2020)

## Built on by (most cited first, 5 of 5)

- [The Best Instruction-Tuning Data are Those That Fit](https://mlsanity.com/paper/2502.04194/) (2025)
- [Formal Disco: Scalable Open-Ended Generation of Formally Verified Programs](https://mlsanity.com/paper/2607.04631/) (2026)
- [Data Selection Through Iterative Self-Filtering for Vision-Language Settings](https://mlsanity.com/paper/2606.23611/) (2026)
- [Preventing Model Collapse: A Fisher-Rao Perspective on the Dynamics of Training with Synthetic Data](https://mlsanity.com/paper/2609.18878/) (2026)
- [Scope Before You Persist: Preventing Cross-Family Interference in Agent Memory](https://mlsanity.com/paper/2609.29144/) (2026)

## Abstract (arXiv)

In various fields of knowledge creation, including science, new ideas often build on pre-existing information. In this work, we explore this concept within the context of language models. Specifically, we explore the potential of self-training models on their own outputs, akin to how humans learn and build on their previous thoughts and actions. While this approach is intuitively appealing, our research reveals its practical limitations. We find that extended self-training of the GPT-2 model leads to a significant degradation in performance, resulting in repetitive and collapsed token output.