# Tournament of Prompts: Evolving LLM Instructions Through Structured Debates and Elo Ratings

arXiv: https://arxiv.org/abs/2506.00178 · HTML: https://mlsanity.com/paper/2506.00178/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 64 references that are in the map.
- First posted on arXiv 30 May 2025.

## Builds on (most cited first, 30 of 30)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023)
- [ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate](https://mlsanity.com/paper/2308.07201/) (2023)
- [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023)
- [Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution](https://mlsanity.com/paper/2309.16797/) (2023)
- [Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge](https://mlsanity.com/paper/2410.02736/) (2024)
- [RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning](https://mlsanity.com/paper/2205.12548/) (2022)
- [Debating with More Persuasive LLMs Leads to More Truthful Answers](https://mlsanity.com/paper/2402.06782/) (2024)
- [Black-Box Tuning for Language-Model-as-a-Service](https://mlsanity.com/paper/2201.03514/) (2022)
- [WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild](https://mlsanity.com/paper/2406.04770/) (2024)
- [Black-Box Prompt Optimization: Aligning Large Language Models without Model Training](https://mlsanity.com/paper/2311.04155/) (2023)
- [Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2108.13161/) (2021)
- [EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithms](https://mlsanity.com/paper/2406.14228/) (2024)
- [Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2104.00783/) (2021)
- [Elo Uncovered: Robustness and Best Practices in Language Model Evaluation](https://mlsanity.com/paper/2311.17295/) (2023)
- [Re-evaluating Evaluation](https://mlsanity.com/paper/1806.02643/) (2018)
- [Into the Unknown Unknowns: Engaged Human Learning through Participation in Language Model Agent Conversations](https://mlsanity.com/paper/2408.15232/) (2024)
- [Finding Generalizable Evidence by Learning to Convince Q&A Models](https://mlsanity.com/paper/1909.05863/) (2019)
- [Efficient and Accurate Prompt Optimization: the Benefit of Memory in Exemplar-Guided Reflection](https://mlsanity.com/paper/2411.07446/) (2024)
- [DEBATE: Devil's Advocate-Based Assessment and Text Evaluation](https://mlsanity.com/paper/2405.09935/) (2024)
- [A Statistical Framework for Ranking LLM-Based Chatbots](https://mlsanity.com/paper/2412.18407/) (2024)
- [DebFlow: Automating Agent Creation via Agent Debate](https://mlsanity.com/paper/2503.23781/) (2025)
- [Prompt Tuning Strikes Back: Customizing Foundation Models with Low-Rank Prompt Adaptation](https://mlsanity.com/paper/2405.15282/) (2024)
- [Concentrate Attention: Towards Domain-Generalizable Prompt Optimization for Language Models](https://mlsanity.com/paper/2406.10584/) (2024)
- [ZeroSumEval: An Extensible Framework For Scaling LLM Evaluation with Inter-Model Competition](https://mlsanity.com/paper/2503.10673/) (2025)
- [Is Elo Rating Reliable? A Study Under Model Misspecification](https://mlsanity.com/paper/2502.10985/) (2025)
- [ELO-Rated Sequence Rewards: Advancing Reinforcement Learning Models](https://mlsanity.com/paper/2409.03301/) (2024)

## Built on by (most cited first, 5 of 5)

- [Where Did It All Go Wrong? A Hierarchical Look into Multi-Agent Error Attribution](https://mlsanity.com/paper/2510.04886/) (2025)
- [Attention-MoA: Enhancing Mixture-of-Agents via Inter-Agent Semantic Attention and Deep Residual Synthesis](https://mlsanity.com/paper/2601.16596/) (2026)
- [UPA: Unsupervised Prompt Agent via Tree-Based Search and Selection](https://mlsanity.com/paper/2601.23273/) (2026)
- [Optimizing for Persuasion Improves LLM Generalization: Evidence from Quality-Diversity Evolution of Debate Strategies](https://mlsanity.com/paper/2510.05909/) (2025)
- [Self-Supervised Skill Optimization](https://mlsanity.com/paper/2607.28777/) (2026)

## Abstract (arXiv)

Prompt engineering represents a critical bottleneck to harness the full potential of Large Language Models (LLMs) for solving complex tasks, as it requires specialized expertise, significant trial-and-error, and manual intervention. This challenge is particularly pronounced for tasks involving subjective quality assessment, where defining explicit optimization objectives becomes fundamentally problematic. Existing automated prompt optimization methods falter in these scenarios, as they typically require well-defined task-specific numerical fitness functions or rely on generic templates that cannot capture the nuanced requirements of complex use cases. We introduce DEEVO (DEbate-driven EVOlutionary prompt optimization), a novel framework that guides prompt evolution through a debate-driven evaluation with an Elo-based selection. Contrary to prior work, DEEVOs approach enables exploration of the discrete prompt space while preserving semantic coherence through intelligent crossover and strategic mutation operations that incorporate debate-based feedback, combining elements from both successful and unsuccessful prompts based on identified strengths rather than arbitrary splicing. Using Elo ratings as a fitness proxy, DEEVO simultaneously drives improvement and preserves valuable diversity in the prompt population. Experimental results demonstrate that DEEVO significantly outperforms both manual prompt engineering and alternative state-of-the-art optimization approaches on open-ended tasks and close-ended tasks despite using no ground truth feedback. By connecting LLMs reasoning capabilities with adaptive optimization, DEEVO represents a significant advancement in prompt optimization research by eliminating the need of predetermined metrics to continuously improve AI systems.