mlsanityData to 14 Oct 2026

Multiple Weaks Win Single Strong: Large Language Models Ensemble Weak Reinforcement Learning Agents into a Supreme One

arXiv 2505.15306 ↗First posted 21 May 2025cs.LG cs.AIcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Model ensemble is a useful approach in reinforcement learning (RL) for training effective agents. Despite wide success of RL, training effective agents remains difficult due to the multitude of factors requiring careful tuning, such as algorithm selection, hyperparameter settings, and even random seed choices, all of which can significantly influence an agent's performance. Model ensemble helps overcome this challenge by combining multiple weak agents into a single, more powerful one, enhancing overall performance. However, existing ensemble methods, such as majority voting and Boltzmann addition, are designed as fixed strategies and lack a semantic understanding of specific tasks, limiting their adaptability and effectiveness. To address this, we propose LLM-Ens, a novel approach that enhances RL model ensemble with task-specific semantic understandings driven by large language models (LLMs). Given a task, we first design an LLM to categorize states in this task into distinct 'situations', incorporating high-level descriptions of the task conditions. Then, we statistically analyze the strengths and weaknesses of each individual agent to be used in the ensemble in each situation. During the inference time, LLM-Ens dynamically identifies the changing task situation and switches to the agent that performs best in the current situation, ensuring dynamic model selection in the evolving task condition. Our approach is designed to be compatible with agents trained with different random seeds, hyperparameter settings, and various RL algorithms. Extensive experiments on the Atari benchmark show that LLM-Ens significantly improves the RL model ensemble, surpassing well-known baselines by up to 20.9%. For reproducibility, our code is open-source at https://anonymous.4open.science/r/LLM4RLensemble-F7EE.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 29 of its 47 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

29 of 47 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Large Language Model as a Policy Teacher for Training Reinforcement Learning Agents22 Nov

Other references · 28

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2023A Survey of Large Language Modelscited by 2,576
  7. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  8. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  9. 2019Model-Based Reinforcement Learning for Ataricited by 502
  10. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 293
  11. 2021Ensemble deep learning: A reviewcited by 284
  12. 2021Mastering Atari Games with Limited Datacited by 237
16 more
  1. 2018An Introduction to Deep Reinforcement Learningcited by 206
  2. 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learningcited by 160
  3. 2024Evaluation of OpenAI o1: Opportunities and Challenges of AGIcited by 127
  4. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 116
  5. 2023A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4cited by 81
  6. 2023Language Instructed Reinforcement Learning for Human-AI Coordinationcited by 63
  7. 2018How Many Random Seeds? Statistical Power Analysis in Deep Reinforcement Learning Experimentscited by 60
  8. 2023Stance Detection with Collaborative Role-Infused LLM-Based Agentscited by 53
  9. 2023Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manualscited by 45
  10. 2023Self-Refined Large Language Model as Automated Reward Function Designer for Deep Reinforcement Learning in Roboticscited by 36
  11. 2023Augmenting Autotelic Agents with Large Language Modelscited by 30
  12. 2024LLM-Empowered State Representation for Reinforcement Learningcited by 28
  13. 2023Ensemble Reinforcement Learning: A Surveycited by 16
  14. 2024HLM-Cite: Hybrid Language Model Workflow for Text-based Scientific Citation Predictioncited by 15
  15. 2024A Method for Evaluating Hyperparameter Sensitivity in Reinforcement Learningcited by 15
  16. 2020Reinforcement Learning based dynamic weighing of Ensemble Models for Time Series Forecastingcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025RLAE: Reinforcement Learning-Assisted Ensemble for LLMs
  2. 2024The RL/LLM Taxonomy Tree: Reviewing Synergies Between Reinforcement Learning and Large Language Models
  3. 2024Training Agents with Weakly Supervised Feedback from Large Language Models
  4. 2025ToolExpander: Extending the Frontiers of Tool-Using Reinforcement Learning to Weak LLMs
  5. 2025Tandem Training for Language Models
  6. 2025EMORL: Ensemble Multi-Objective Reinforcement Learning for Efficient and Flexible LLM Fine-Tuning
  7. 2025Dynamic Optimizations of LLM Ensembles with Two-Stage Reinforcement Learning Agents
  8. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs