mlsanityData to 7 Oct 2026

ATLAS: A Multi-LLM Training Framework for EvoDPO with Adaptive Reference Evolution

arXiv 2602.02709 ↗First posted 2 Feb 2026cs.AIcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Recent multi-LLM agent systems have shown promising capabilities for automated problem-solving, yet they predominantly rely on frozen agents or static fine-tuning pipelines. To address this limitation, our primary contribution is ATLAS (Adaptive Task-distributed Learning for Agentic Self-evolution), a multi-agent framework where specialized meta-agents collaboratively train and refine an active agent toward a domain-specific policy. A core challenge in iterative preference learning within these pipelines is the reliance on fixed reference models, which typically leads to overly conservative updates or training stagnation. To overcome this, the framework's algorithmic engine utilizes Evolving Direct Preference Optimization (EvoDPO). EvoDPO employs an inspection agent to perform adaptive, proxy-KL gated reference policy updates based on continuous training telemetry. We evaluate this full framework across a diverse set of challenging environments-including non-stationary contextual bandits, partial differential equations (PINNs), and combinatorial optimization tasks (TSP, Bin Packing). Through comparison against fixed-reference, adaptive-reference, and external automated-discovery baselines, our results suggest that ATLAS combines supporter-driven exploration with EvoDPO-driven stability to improve long-horizon evaluator-driven self-improvement.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 12 of its 32 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

12 of 32 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024MACPO: Weak-to-Strong Alignment via Multi-Agent Contrastive Preference Optimization10 Oct

Other references · 11

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  3. 2023AutoAgents: A Framework for Automatic Agent Generationcited by 211
  4. 2025Algorithm Discovery With LLMs: Evolutionary Search Meets Reinforcement Learningcited by 39
  5. 2024TPO: Aligning Large Language Models with Multi-branch & Multi-step Preference Treescited by 19
  6. 2024Weighted-Reward Preference Optimization for Implicit Model Fusioncited by 15
  7. 2024AI Metropolis: Scaling Large Language Model-based Multi-Agent Simulation with Out-of-order Executioncited by 14
  8. 2025AgenticSciML: Collaborative Multi-Agent Systems for Emergent Discovery in Scientific Machine Learningcited by 13
  9. 2025Debate, Reflect, and Distill: Multi-Agent Feedback with Tree-Structured Preference Optimization for Efficient Language Model Enhancementcited by 10
  10. 2025Automated Code Development for PDE Solvers Using Large Language Modelscited by 5
  11. 2025WarriorMath: Enhancing the Mathematical Ability of Large Language Models with a Defect-aware Frameworkcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026EvoTrainer: Co-Evolving LLM Policies and Training Harnesses for Autonomous Agentic Reinforcement Learning
  2. 2026Self-evolving LLM agents with in-distribution Optimization
  3. 2025Multi-Agent Evolve: LLM Self-Improve through Co-evolution
  4. 2026LLM-ALSO: LLM-Driven Adaptive Learning-Signal Optimization for Multi-Agent Reinforcement Learning
  5. 2026CoEvolve: Training LLM Agents via Agent-Data Mutual Evolution
  6. 2026Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learning
  7. 2026HarnessForge: Joint Harness and Policy Evolution for Adaptive Agent Systems
  8. 2025Learn as Individuals, Evolve as a Team: Multi-agent LLMs Adaptation in Embodied Environments