# Active Evaluation: Efficient NLG Evaluation with Few Pairwise Comparisons

arXiv: https://arxiv.org/abs/2203.06063 · HTML: https://mlsanity.com/paper/2203.06063/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 73 references that are in the map.
- First posted on arXiv 11 Mar 2022.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://mlsanity.com/paper/1812.10464/) (2018)
- [MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance](https://mlsanity.com/paper/1909.02622/) (2019)
- [Why We Need New Evaluation Metrics for NLG](https://mlsanity.com/paper/1707.06875/) (2017)
- [A Survey of Evaluation Metrics Used for NLG Systems](https://mlsanity.com/paper/2008.12009/) (2020)
- [What makes a good conversation? How controllable attributes affect human judgments](https://mlsanity.com/paper/1902.08654/) (2019)
- [Handling Divergent Reference Texts when Evaluating Table-to-Text Generation](https://mlsanity.com/paper/1906.01081/) (2019)
- [Preference-based Online Learning with Dueling Bandits: A Survey](https://mlsanity.com/paper/1807.11398/) (2018)
- [Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge](https://mlsanity.com/paper/1901.07931/) (2019)
- [ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons](https://mlsanity.com/paper/1909.03087/) (2019)
- [RankME: Reliable Human Ratings for Natural Language Generation](https://mlsanity.com/paper/1803.05928/) (2018)
- [Towards Universal Paraphrastic Sentence Embeddings](https://mlsanity.com/paper/1511.08198/) (2015)
- [Improving Dialog Evaluation with a Multi-reference Adversarial Dataset and Large Scale Pretraining](https://mlsanity.com/paper/2009.11321/) (2020)
- [Regret Lower Bound and Optimal Algorithm in Dueling Bandit Problem](https://mlsanity.com/paper/1506.02550/) (2015)
- [Importance of Search and Evaluation Strategies in Neural Dialogue Modeling](https://mlsanity.com/paper/1811.00907/) (2018)
- [Double Thompson Sampling for Dueling Bandits](https://mlsanity.com/paper/1604.07101/) (2016)
- [ParaBank: Monolingual Bitext Generation and Sentential Paraphrasing via Lexically-constrained Neural Machine Translation](https://mlsanity.com/paper/1901.03644/) (2019)
- [Copeland Dueling Bandits](https://mlsanity.com/paper/1506.00312/) (2015)
- [Efficient Online Scalar Annotation with Bounded Support](https://mlsanity.com/paper/1806.01170/) (2018)
- [Re-evaluating ADEM: A Deeper Look at Scoring Dialogue Responses](https://mlsanity.com/paper/1902.08832/) (2019)
- [Beyond User Self-Reported Likert Scale Ratings: A Comparison Model for Automatic Dialog Evaluation](https://mlsanity.com/paper/2005.10716/) (2020)
- [Maximum Selection and Ranking under Noisy Comparisons](https://mlsanity.com/paper/1705.05366/) (2017)

## Built on by (most cited first, 10 of 10)

- [A Survey of Active Learning for Natural Language Processing](https://mlsanity.com/paper/2210.10109/) (2022)
- [Label-Efficient Model Selection for Text Generation](https://mlsanity.com/paper/2402.07891/) (2024)
- [The Road to Artificial SuperIntelligence: A Comprehensive Survey of Superalignment](https://mlsanity.com/paper/2412.16468/) (2024)
- [On the Effectiveness of Automated Metrics for Text Generation Systems](https://mlsanity.com/paper/2210.13025/) (2022)
- [Better than Random: Reliable NLG Human Evaluation with Constrained Active Sampling](https://mlsanity.com/paper/2406.07967/) (2024)
- [EZ-Sort: Efficient Pairwise Comparison via Zero-Shot CLIP-Based Pre-Ordering and Human-in-the-Loop Sorting](https://mlsanity.com/paper/2508.21550/) (2025)
- [Offline Preference-Based Trajectory Evaluation](https://mlsanity.com/paper/2606.17541/) (2026)
- [Baby Bear: Seeking a Just Right Rating Scale for Scalar Annotations](https://mlsanity.com/paper/2408.09765/) (2024)
- [Research Superalignment Should Advance Now with Alternating Competence and Conformity Optimization](https://mlsanity.com/paper/2503.07660/) (2025)
- [Dynamically Allocating Evaluation Effort for Model Ranking](https://mlsanity.com/paper/2608.03437/) (2026)

## Abstract (arXiv)

Recent studies have shown the advantages of evaluating NLG systems using pairwise comparisons as opposed to direct assessment. Given k systems, a naive approach for identifying the top-ranked system would be to uniformly obtain pairwise comparisons from all k choose 2 pairs of systems. However, this can be very expensive as the number of human annotations required would grow quadratically with k. In this work, we introduce Active Evaluation, a framework to efficiently identify the top-ranked system by actively choosing system pairs for comparison using dueling bandit algorithms. We perform extensive experiments with 13 dueling bandits algorithms on 13 NLG evaluation datasets spanning 5 tasks and show that the number of human annotations can be reduced by 80%. To further reduce the number of human annotations, we propose model-based dueling bandit algorithms which combine automatic evaluation metrics with human evaluations. Specifically, we eliminate sub-optimal systems even before the human annotation process and perform human evaluations only on test examples where the automatic metric is highly uncertain. This reduces the number of human annotations required further by 89%. In effect, we show that identifying the top-ranked system requires only a few hundred human annotations, which grow linearly with k. Lastly, we provide practical recommendations and best practices to identify the top-ranked system efficiently. Our code has been made publicly available at https://github.com/akashkm99/duelnlg