mlsanityData to 7 Oct 2026

Elo-Evolve: A Co-evolutionary Framework for Language Model Alignment

arXiv 2602.13575 ↗First posted 14 Feb 2026cs.CL cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Current alignment methods for Large Language Models (LLMs) rely on compressing vast amounts of human preference data into static, absolute reward functions, leading to data scarcity, noise sensitivity, and training instability. We introduce Elo-Evolve, a co-evolutionary framework that redefines alignment as dynamic multi-agent competition within an adaptive opponent pool. Our approach makes two key innovations: (1) eliminating Bradley-Terry model dependencies by learning directly from binary win/loss outcomes in pairwise competitions, and (2) implementing Elo-orchestrated opponent selection that provides automatic curriculum learning through temperature-controlled sampling. We ground our approach in PAC learning theory, demonstrating that pairwise comparison achieves superior sample complexity and empirically validate a 4.5x noise reduction compared to absolute scoring approaches. Experimentally, we train a Qwen2.5-7B model using our framework with opponents including Qwen2.5-14B, Qwen2.5-32B, and Qwen3-8B models. Results demonstrate a clear performance hierarchy: point-based methods < static pairwise training < Elo-Evolve across Alpaca Eval 2.0 and MT-Bench, validating the progressive benefits of pairwise comparison and dynamic opponent selection for LLM alignment.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 19 of its 25 references that are in the map. First posted on arXiv 14 Feb 2026.

Builds on

19 of 25 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Relative Preference Optimization: Enhancing LLM Alignment through Contrasting Responses across Identical and Diverse Prompts12 Feb
  2. 2024LiPO: Listwise Preference Optimization through Learning-to-Rank2 Feb
  3. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 16

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2024Qwen2.5 Technical Reportcited by 4,713
  7. 2017Deep reinforcement learning from human preferencescited by 3,846
  8. 2020Learning to summarize from human feedbackcited by 2,562
  9. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  10. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  11. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  12. 2023UltraFeedback: Boosting Language Models with Scaled AI Feedbackcited by 394
4 more
  1. 2024Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferencescited by 176
  2. 2025Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learningcited by 88
  3. 2025J1: Incentivizing Thinking in LLM-as-a-Judge via Reinforcement Learningcited by 73
  4. 2024Rethinking Bradley-Terry Models in Preference-Based Reward Modeling: Foundations, Theory, and Alternativescited by 48

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025ESSA: Evolutionary Strategies for Scalable Alignment
  2. 2026EvoPref: Multi-Objective Evolutionary Optimization Discovers Diverse LLM Alignments Beyond Gradient Descent
  3. 2024Sample-Efficient Alignment for LLMs
  4. 2025Evolving Language Models without Labels: Majority Drives Selection, Novelty Promotes Variation
  5. 2024Self-Exploring Language Models: Active Preference Elicitation for Online Alignment
  6. 2026Multi-LLM Collaborative Alignment via Stackelberg Games
  7. 2024DeAL: Decoding-time Alignment for Large Language Models
  8. 2025Aligning Frozen LLMs by Reinforcement Learning: An Iterative Reweight-then-Optimize Approach