mlsanityData to 14 Oct 2026

TeamTR: Trust-Region Fine-Tuning for Multi-Agent LLM Coordination

arXiv 2605.15207 ↗First posted 1 May 2026cs.LG cs.MAcited by 1

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Multi-agent LLM systems have shown promise for complex reasoning, yet recent evaluations reveal they often underperform single-model baselines. We identify a structural failure mode in sequential fine-tuning of shared-context teams: updating one agent shifts the team's context distribution, and when subsequent updates are evaluated on cached rollouts, this mismatch compounds. We formalize this as the compounding occupancy shift and prove that stale-occupancy evaluation incurs a penalty that scales quadratically with the number of agents. In contrast, intermediate-occupancy evaluation reduces this to linear scaling. We propose TeamTR, a trust-region framework that resamples trajectories after each component update and enforces per-agent divergence control, yielding rigorous per-update and per-stage improvement lower bounds. Experiments show that TeamTR outperforms single-agent and sequential baselines with 7.1% on average, mitigates coordination regressions, and supports plug-and-play component replacement. Code is available at https://github.com/Yydc/TeamTR.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 50 of its 60 references that are in the map. First posted on arXiv 1 May 2026.

Builds on

50 of 60 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Towards a Science of Scaling Agent Systems9 Dec
  2. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMs19 Oct
  3. 2023FireAct: Toward Language Agent Fine-tuning9 Oct
  4. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug
  5. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society31 Mar

Other references · 45

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2025Qwen3 Technical Reportcited by 8,820
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  9. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  10. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  11. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  12. 2015Trust Region Policy Optimizationcited by 2,580
33 more
  1. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  2. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  3. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  4. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  5. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  6. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  7. 2022Editing Models with Task Arithmeticcited by 1,120
  8. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  9. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
  10. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  11. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  12. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  13. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
  14. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
  15. 2025Why Do Multi-Agent LLM Systems Fail?cited by 604
  16. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 564
  17. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
  18. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 382
  19. 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 364
  20. 2023A Long Way to Go: Investigating Length Correlations in RLHFcited by 291
  21. 2025ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoningcited by 201
  22. 2023Simple synthetic data reduces sycophancy in large language modelscited by 187
  23. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 186
  24. 2017Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learningcited by 117
  25. 2023Shepherd: A Critic for Language Model Generationcited by 97
  26. 2025Multiagent Finetuning: Self Improvement with Diverse Reasoning Chainscited by 95
  27. 2024Self-Consistency Preference Optimizationcited by 50
  28. 2025LLM Collaboration With Multi-Agent Reinforcement Learningcited by 49
  29. 2025AutoLogi: Automated Generation of Logic Puzzles for Evaluating Reasoning Abilities of Large Language Modelscited by 31
  30. 2025From Debate to Equilibrium: Belief-Driven Multi-Agent LLM Reasoning via Bayesian Nash Equilibriumcited by 26
  31. 2025From Passive to Active Reasoning: Can Large Language Models Ask the Right Questions under Incomplete Information?cited by 23
  32. 2023Improving Generalization in Visual Reinforcement Learning via Conflict-aware Gradient Agreement Augmentationcited by 14
  33. 2026SAT: Sequential Agent Tuning for Coordinator Free Plug and Play Multi-LLM Training with Monotonic Improvement Guaranteescited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026DICE: Entropy-Regularized Equilibrium Selection for Stable Multi-Agent LLM Coordinationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025COMPASS: Enhancing Agent Long-Horizon Reasoning with Evolving Context
  2. 2026Multi-agent Scaling Across Disjunctive and Compensatory Tasks
  3. 2026SyncPlan: Long-Horizon LLM Coordination with Explicit Synchronization and Adaptive Correction
  4. 2025SagaLLM: Context Management, Validation, and Transaction Guarantees for Multi-Agent LLM Planning
  5. 2026Competence-Aware Orchestration for LLM-Based Multi-Agent Systems
  6. 2026Team of Thoughts: Efficient Test-time Scaling of Agentic Systems through Orchestrated Tool Calling
  7. 2026DPBench: Structural Determinants of Multi-Agent LLM Coordination Under Simultaneous Resource Contention
  8. 2026Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systems