# On the Relation between Quality-Diversity Evaluation and Distribution-Fitting Goal in Text Generation

arXiv: https://arxiv.org/abs/2007.01488 · HTML: https://mlsanity.com/paper/2007.01488/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jul 2020.

## Built on by (most cited first, 7 of 7)

- [On the Diversity of Synthetic Data and its Impact on Training Large Language Models](https://mlsanity.com/paper/2410.15226/) (2024)
- [LitBench: A Benchmark and Dataset for Reliable Evaluation of Creative Writing](https://mlsanity.com/paper/2507.00769/) (2025)
- [Fuse It More Deeply! A Variational Transformer with Layer-Wise Latent Variable Inference for Text Generation](https://mlsanity.com/paper/2207.06130/) (2022)
- [Evade the Trap of Mediocrity: Promoting Diversity and Novelty in Text Generation via Concentrating Attention](https://mlsanity.com/paper/2211.07164/) (2022)
- [Recurrence Boosts Diversity! Revisiting Recurrent Latent Variable in Transformer-Based Variational AutoEncoder for Diverse Text Generation](https://mlsanity.com/paper/2210.12409/) (2022)
- [Distributional Open-Ended Evaluation of LLM Cultural Value Alignment Based on Value Codebook](https://mlsanity.com/paper/2604.06210/) (2026)
- [Distribution Learning Based on Evolutionary Algorithm Assisted Deep Neural Networks for Imbalanced Image Classification](https://mlsanity.com/paper/2207.12744/) (2022)

## Abstract (arXiv)

The goal of text generation models is to fit the underlying real probability distribution of text. For performance evaluation, quality and diversity metrics are usually applied. However, it is still not clear to what extend can the quality-diversity evaluation reflect the distribution-fitting goal. In this paper, we try to reveal such relation in a theoretical approach. We prove that under certain conditions, a linear combination of quality and diversity constitutes a divergence metric between the generated distribution and the real distribution. We also show that the commonly used BLEU/Self-BLEU metric pair fails to match any divergence metric, thus propose CR/NRR as a substitute for quality/diversity metric pair.