# A Judge-free LLM Open-ended Generation Benchmark Based on the Distributional Hypothesis

arXiv: https://arxiv.org/abs/2502.09316 · HTML: https://mlsanity.com/paper/2502.09316/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Feb 2025.

## Built on by (most cited first, 4 of 4)

- [PLaMo 2 Technical Report](https://mlsanity.com/paper/2509.04897/) (2025)
- [CRACQ: A Multi-Dimensional Approach To Automated Document Assessment](https://mlsanity.com/paper/2510.02337/) (2025)
- [Naturalistic measure of social norms alignment](https://mlsanity.com/paper/2605.23420/) (2026)
- [Automatic Hard Example Synthesis with Multi-Level Agentic Data Curation](https://mlsanity.com/paper/2607.14256/) (2026)

## Abstract (arXiv)

Evaluating the open-ended text generation of large language models (LLMs) is challenging because of the lack of a clear ground truth and the high cost of human or LLM-based assessments. We propose a novel benchmark that evaluates LLMs using n-gram statistics and rules, without relying on human judgement or LLM-as-a-judge approaches. Using 50 question and reference answer sets, we introduce three new metrics based on n-grams and rules: Fluency, Truthfulness, and Helpfulness. Our benchmark strongly correlates with GPT-4o-based evaluations while requiring significantly fewer computational resources, demonstrating its effectiveness as a scalable alternative for assessing LLMs' open-ended generation capabilities.