# TreeEval: Benchmark-Free Evaluation of Large Language Models through Tree Planning

arXiv: https://arxiv.org/abs/2402.13125 · HTML: https://mlsanity.com/paper/2402.13125/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 42 references that are in the map.
- First posted on arXiv 20 Feb 2024.

## Builds on (most cited first, 29 of 29)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models](https://mlsanity.com/paper/2304.06364/) (2023)
- [Zephyr: Direct Distillation of LM Alignment](https://mlsanity.com/paper/2310.16944/) (2023)
- [JudgeLM: Fine-tuned Large Language Models are Scalable Judges](https://mlsanity.com/paper/2310.17631/) (2023)
- [GPTScore: Evaluate as You Desire](https://mlsanity.com/paper/2302.04166/) (2023)
- [NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark](https://mlsanity.com/paper/2310.18018/) (2023)
- [PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization](https://mlsanity.com/paper/2306.05087/) (2023)
- [Secrets of RLHF in Large Language Models Part I: PPO](https://mlsanity.com/paper/2307.04964/) (2023)
- [Large Language Model Guided Tree-of-Thought](https://mlsanity.com/paper/2305.08291/) (2023)
- [Rethinking Benchmark and Contamination for Language Models with Rephrased Samples](https://mlsanity.com/paper/2311.04850/) (2023)
- [Don't Make Your LLM an Evaluation Benchmark Cheater](https://mlsanity.com/paper/2311.01964/) (2023)
- [Benchmarking Foundation Models with Language-Model-as-an-Examiner](https://mlsanity.com/paper/2306.04181/) (2023)
- [Secrets of RLHF in Large Language Models Part II: Reward Modeling](https://mlsanity.com/paper/2401.06080/) (2024)
- [Generative Judge for Evaluating Alignment](https://mlsanity.com/paper/2310.05470/) (2023)
- [Branch-Solve-Merge Improves Large Language Model Evaluation and Generation](https://mlsanity.com/paper/2310.15123/) (2023)
- [DyVal: Dynamic Evaluation of Large Language Models for Reasoning Tasks](https://mlsanity.com/paper/2309.17167/) (2023)
- [Wider and Deeper LLM Networks are Fairer LLM Evaluators](https://mlsanity.com/paper/2308.01862/) (2023)
- [Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark](https://mlsanity.com/paper/2305.14938/) (2023)
- [ChatGPT's One-year Anniversary: Are Open-Source Large Language Models Catching up?](https://mlsanity.com/paper/2311.16989/) (2023)

## Built on by (most cited first, 14 of 14)

- [Benchmark Data Contamination of Large Language Models: A Survey](https://mlsanity.com/paper/2406.04244/) (2024)
- [AutoBencher: Towards Declarative Benchmark Construction](https://mlsanity.com/paper/2407.08351/) (2024)
- [Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluation](https://mlsanity.com/paper/2502.17521/) (2025)
- [A Survey on Data Contamination for Large Language Models](https://mlsanity.com/paper/2502.14425/) (2025)
- [Do These LLM Benchmarks Agree? Fixing Benchmark Evaluation with BenchBench](https://mlsanity.com/paper/2407.13696/) (2024)
- [DARG: Dynamic Evaluation of Large Language Models via Adaptive Reasoning Graph](https://mlsanity.com/paper/2406.17271/) (2024)
- [Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination](https://mlsanity.com/paper/2503.04149/) (2025)
- [NetArena: Dynamic Benchmarks for AI Agents in Network Automation](https://mlsanity.com/paper/2506.03231/) (2025)
- [Grounding Synthetic Data Evaluations of Language Models in Unsupervised Document Corpora](https://mlsanity.com/paper/2505.08905/) (2025)
- [LiveFact: A Dynamic, Time-Aware Benchmark for LLM-Driven Fake News Detection](https://mlsanity.com/paper/2604.04815/) (2026)
- [RECKON: Large-scale Reference-based Efficient Knowledge Evaluation for Large Language Model](https://mlsanity.com/paper/2504.00756/) (2025)
- [LNE-Blocking: An Efficient Framework for Contamination Mitigation Evaluation on Large Language Models](https://mlsanity.com/paper/2509.15218/) (2025)
- [Silicon Bureaucracy and AI Test-Oriented Education: Contamination Sensitivity and Score Confidence in LLM Benchmarks](https://mlsanity.com/paper/2603.21636/) (2026)
- [Zero Gap Is Not Restoration: Stratified Per-Question Probability Evaluation and Step-wise Mitigation of Benchmark Contamination](https://mlsanity.com/paper/2608.07341/) (2026)

## Abstract (arXiv)

Recently, numerous new benchmarks have been established to evaluate the performance of large language models (LLMs) via either computing a holistic score or employing another LLM as a judge. However, these approaches suffer from data leakage due to the open access of the benchmark and inflexible evaluation process. To address this issue, we introduce TreeEval, a benchmark-free evaluation method for LLMs that let a high-performance LLM host an irreproducible evaluation session and essentially avoids the data leakage. Moreover, this LLM performs as an examiner to raise up a series of questions under a topic with a tree planing strategy, which considers the current evaluation status to decide the next question generation and ensures the completeness and efficiency of the evaluation process. We evaluate 6 models of different parameter sizes, including 7B, 13B, and 33B, and ultimately achieved the highest correlation coefficient with AlpacaEval2.0 using only around 45 questions. We also conduct more analysis to show the robustness and reliability of TreeEval. Our code can be accessed via the provided https://github.com/Ashura5/TreeEval.