# Improving Maximum Likelihood Training for Text Generation with Density Ratio Estimation

arXiv: https://arxiv.org/abs/2007.06018 · HTML: https://mlsanity.com/paper/2007.06018/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jul 2020.

## Built on by (most cited first, 5 of 5)

- [Direct Judgement Preference Optimization](https://mlsanity.com/paper/2409.14664/) (2024)
- [Tailoring Language Generation Models under Total Variation Distance](https://mlsanity.com/paper/2302.13344/) (2023)
- [Generative Cooperative Networks for Natural Language Generation](https://mlsanity.com/paper/2201.12320/) (2022)
- [Triangular Bidword Generation for Sponsored Search Auction](https://mlsanity.com/paper/2101.11349/) (2021)
- [Generating Self-Contained and Summary-Centric Question Answer Pairs via Differentiable Reward Imitation Learning](https://mlsanity.com/paper/2109.04689/) (2021)

## Abstract (arXiv)

Auto-regressive sequence generative models trained by Maximum Likelihood Estimation suffer the exposure bias problem in practical finite sample scenarios. The crux is that the number of training samples for Maximum Likelihood Estimation is usually limited and the input data distributions are different at training and inference stages. Many method shave been proposed to solve the above problem (Yu et al., 2017; Lu et al., 2018), which relies on sampling from the non-stationary model distribution and suffers from high variance or biased estimations. In this paper, we proposepsi-MLE, a new training scheme for auto-regressive sequence generative models, which is effective and stable when operating at large sample space encountered in text generation. We derive our algorithm from a new perspective of self-augmentation and introduce bias correction with density ratio estimation. Extensive experimental results on synthetic data and real-world text generation tasks demonstrate that our method stably outperforms Maximum Likelihood Estimation and other state-of-the-art sequence generative models in terms of both quality and diversity.