# DEBATE, TRAIN, EVOLVE: Self Evolution of Language Model Reasoning

arXiv: https://arxiv.org/abs/2505.15734 · HTML: https://mlsanity.com/paper/2505.15734/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 14 of its 33 references that are in the map.
- First posted on arXiv 21 May 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 14 of 14)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Phi-4 Technical Report](https://mlsanity.com/paper/2412.08905/) (2024)
- [Phi-4-reasoning Technical Report](https://mlsanity.com/paper/2504.21318/) (2025)
- [Verbosity Bias in Preference Labeling by Large Language Models](https://mlsanity.com/paper/2310.10076/) (2023)
- [Towards Reasoning Ability of Small Language Models](https://mlsanity.com/paper/2502.11569/) (2025)
- [Automatic Instruction Evolving for Large Language Models](https://mlsanity.com/paper/2406.00770/) (2024)
- [SELF: Self-Evolution with Language Feedback](https://mlsanity.com/paper/2310.00533/) (2023)
- [Self-Refine Instruction-Tuning for Aligning Reasoning in Language Models](https://mlsanity.com/paper/2405.00402/) (2024)
- [ACC-Collab: An Actor-Critic Approach to Multi-Agent LLM Collaboration](https://mlsanity.com/paper/2411.00053/) (2024)
- [Think, Prune, Train, Improve: Scaling Reasoning without Scaling Models](https://mlsanity.com/paper/2504.18116/) (2025)

## Built on by (most cited first, 6 of 6)

- [Latent Agents: A Post-Training Procedure for Internalized Multi-Agent Debate](https://mlsanity.com/paper/2604.24881/) (2026)
- [Multi-Agent Debate Strategies: Survey, Taxonomy, and Challenges](https://mlsanity.com/paper/2607.26212/) (2026)
- [Phantom Gains: Auditing Self-Improvement Against a Measured Null](https://mlsanity.com/paper/2608.20290/) (2026)
- [Multiperspectivity as a Resource for Narrative Similarity Prediction](https://mlsanity.com/paper/2603.22103/) (2026)
- [Ask-E: An Environment for Calibrated Question Generation](https://mlsanity.com/paper/2608.06933/) (2026)
- [Unsupervised Post-Training of Foundation Models: A Survey](https://mlsanity.com/paper/2608.24982/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have improved significantly in their reasoning through extensive training on massive datasets. However, relying solely on additional data for improvement is becoming increasingly impractical, highlighting the need for models to autonomously enhance their reasoning without external supervision. In this paper, we propose Debate, Train, Evolve (DTE), a novel ground truth-free training framework that uses multi-agent debate traces to evolve a single language model. We also introduce a new prompting strategy Reflect-Critique-Refine, to improve debate quality by explicitly instructing agents to critique and refine their reasoning. Extensive evaluations on seven reasoning benchmarks with six open-weight models show that our DTE framework achieve substantial improvements, with an average accuracy gain of 8.92% on the challenging GSM-PLUS dataset. Furthermore, we observe strong cross-domain generalization, with an average accuracy gain of 5.8% on all other benchmarks, suggesting that our method captures general reasoning capabilities. Our framework code and trained models are publicly available at https://github.com/ctrl-gaurav/Debate-Train-Evolve