# Evaluating Text GANs as Language Models

arXiv: https://arxiv.org/abs/1810.12686 · HTML: https://mlsanity.com/paper/1810.12686/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2018.

## Built on by (most cited first, 5 of 5)

- [Pros and Cons of GAN Evaluation Measures: New Developments](https://mlsanity.com/paper/2103.09396/) (2021)
- [Text Generation by Learning from Demonstrations](https://mlsanity.com/paper/2009.07839/) (2020)
- [Survey on Automated Short Answer Grading with Deep Learning: from Word Embeddings to Transformers](https://mlsanity.com/paper/2204.03503/) (2022)
- [Adversarial Machine Learning in Text Analysis and Generation](https://mlsanity.com/paper/2101.08675/) (2021)
- [Why GANs are overkill for NLP](https://mlsanity.com/paper/2205.09838/) (2022)

## Abstract (arXiv)

Generative Adversarial Networks (GANs) are a promising approach for text generation that, unlike traditional language models (LM), does not suffer from the problem of ``exposure bias''. However, A major hurdle for understanding the potential of GANs for text generation is the lack of a clear evaluation metric. In this work, we propose to approximate the distribution of text generated by a GAN, which permits evaluating them with traditional probability-based LM metrics. We apply our approximation procedure on several GAN-based models and show that they currently perform substantially worse than state-of-the-art LMs. Our evaluation procedure promotes better understanding of the relation between GANs and LMs, and can accelerate progress in GAN-based text generation.