# Mark-Evaluate: Assessing Language Generation using Population Estimation Methods

arXiv: https://arxiv.org/abs/2010.04606 · HTML: https://mlsanity.com/paper/2010.04606/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Oct 2020.

## Built on by (most cited first, 5 of 5)

- [Deep Learning on a Healthy Data Diet: Finding Important Examples for Fairness](https://mlsanity.com/paper/2211.11109/) (2022)
- [FrugalScore: Learning Cheaper, Lighter and Faster Evaluation Metricsfor Automatic Text Generation](https://mlsanity.com/paper/2110.08559/) (2021)
- [Should We Attend More or Less? Modulating Attention for Fairness](https://mlsanity.com/paper/2305.13088/) (2023)
- [Evaluating Post-Training Compression in GANs using Locality-Sensitive Hashing](https://mlsanity.com/paper/2103.11912/) (2021)
- [How important is Recall for Measuring Retrieval Quality?](https://mlsanity.com/paper/2512.20854/) (2025)

## Abstract (arXiv)

We propose a family of metrics to assess language generation derived from population estimation methods widely used in ecology. More specifically, we use mark-recapture and maximum-likelihood methods that have been applied over the past several decades to estimate the size of closed populations in the wild. We propose three novel metrics: MEPetersen and MECAPTURE, which retrieve a single-valued assessment, and MESchnabel which returns a double-valued metric to assess the evaluation set in terms of quality and diversity, separately. In synthetic experiments, our family of methods is sensitive to drops in quality and diversity. Moreover, our methods show a higher correlation to human evaluation than existing metrics on several challenging tasks, namely unconditional language generation, machine translation, and text summarization.