# Eval all, trust a few, do wrong to none: Comparing sentence generation models

arXiv: https://arxiv.org/abs/1804.07972 · HTML: https://mlsanity.com/paper/1804.07972/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Apr 2018.

## Built on by (most cited first, 8 of 8)

- [The Vendi Score: A Diversity Evaluation Metric for Machine Learning](https://mlsanity.com/paper/2210.02410/) (2022)
- [How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models](https://mlsanity.com/paper/2102.08921/) (2021)
- [Measuring Diversity in Synthetic Datasets](https://mlsanity.com/paper/2502.08512/) (2025)
- [A Variational AutoEncoder for Transformers with Nonparametric Variational Information Bottleneck](https://mlsanity.com/paper/2207.13529/) (2022)
- [vec2text with Round-Trip Translations](https://mlsanity.com/paper/2209.06792/) (2022)
- [Adversarial Machine Learning in Text Analysis and Generation](https://mlsanity.com/paper/2101.08675/) (2021)
- [Language Model Evaluation in Open-ended Text Generation](https://mlsanity.com/paper/2108.03578/) (2021)
- [MAD Speech: Measures of Acoustic Diversity of Speech](https://mlsanity.com/paper/2404.10419/) (2024)

## Abstract (arXiv)

In this paper, we study recent neural generative models for text generation related to variational autoencoders. Previous works have employed various techniques to control the prior distribution of the latent codes in these models, which is important for sampling performance, but little attention has been paid to reconstruction error. In our study, we follow a rigorous evaluation protocol using a large set of previously used and novel automatic and human evaluation metrics, applied to both generated samples and reconstructions. We hope that it will become the new evaluation standard when comparing neural generative models for text.