# Generative Active Testing: Efficient LLM Evaluation via Proxy Task Adaptation

arXiv: https://arxiv.org/abs/2603.19264 · HTML: https://mlsanity.com/paper/2603.19264/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 28 references that are in the map.
- First posted on arXiv 26 Feb 2026.

## Builds on (most cited first, 20 of 20)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [tinyBenchmarks: evaluating LLMs with fewer examples](https://mlsanity.com/paper/2402.14992/) (2024)
- [Active Testing: Sample-Efficient Model Evaluation](https://mlsanity.com/paper/2103.05331/) (2021)
- [On Statistical Bias In Active Learning: How and When To Fix It](https://mlsanity.com/paper/2101.11665/) (2021)
- [Can NLI Models Verify QA Systems' Predictions?](https://mlsanity.com/paper/2104.08731/) (2021)
- [Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation](https://mlsanity.com/paper/2202.06881/) (2022)
- [Label-Efficient Model Selection for Text Generation](https://mlsanity.com/paper/2402.07891/) (2024)
- [Learning Distinctive Margin toward Active Domain Adaptation](https://mlsanity.com/paper/2203.05738/) (2022)
- [AcTracer: Active Testing of Large Language Model via Multi-Stage Sampling](https://mlsanity.com/paper/2408.03573/) (2024)
- [Are Machines Better at Complex Reasoning? Unveiling Human-Machine Inference Gaps in Entailment Verification](https://mlsanity.com/paper/2402.03686/) (2024)
- [Scaling Up Active Testing to Large Language Models](https://mlsanity.com/paper/2508.09093/) (2025)
- [FlashEval: Towards Fast and Accurate Evaluation of Text-to-image Diffusion Generative Models](https://mlsanity.com/paper/2403.16379/) (2024)
- [Advancing Healthcare Automation: Multi-Agent System for Medical Necessity Justification](https://mlsanity.com/paper/2404.17977/) (2024)
- [Can GPT Improve the State of Prior Authorization via Guideline Based Automated Question Answering?](https://mlsanity.com/paper/2402.18419/) (2024)

## Built on by (most cited first, 2 of 2)

- [Active Testing of Large Language Models via Approximate Neyman Allocation](https://mlsanity.com/paper/2605.10075/) (2026)
- [Are We Testing Too Much? A Survey of Benchmark Compression for Efficient Evaluation of Large Language Models](https://mlsanity.com/paper/2610.18248/) (2026)

## Abstract (arXiv)

With the widespread adoption of pre-trained Large Language Models (LLM), there exists a high demand for task-specific test sets to benchmark their performance in domains such as healthcare and biomedicine. However, the cost of labeling test samples while developing new benchmarks poses a significant challenge, especially when expert annotators are required. Existing frameworks for active sample selection offer limited support for generative Question Answering tasks, where option dynamics can affect model decision boundaries. In this paper, we present Generative Active Testing (GAT), an uncertainty-aware acquisition framework leveraging LLMs as surrogates for informing the sample selection process. Using a novel Statement Adaptation Module, we modify generative tasks into a pseudo-classification format, enabling the capture of sample-level uncertainties across unlabeled candidates. Our zero-shot acquisition functions reduce estimation error by ~40% compared to traditional sampling baselines, offering a scalable solution for cost-effective model benchmarking.