mlsanityData to 14 Oct 2026

Divide-Fuse-Conquer: Eliciting "Aha Moments" in Multi-Scenario Games

arXiv 2505.16401 ↗First posted 22 May 2025cs.LGcited by 4

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Large language models (LLMs) have been observed to suddenly exhibit advanced reasoning abilities during reinforcement learning (RL), resembling an ``aha moment'' triggered by simple outcome-based rewards. While RL has proven effective in eliciting such breakthroughs in tasks involving mathematics, coding, and vision, it faces significant challenges in multi-scenario games. The diversity of game rules, interaction modes, and environmental complexities often leads to policies that perform well in one scenario but fail to generalize to others. Simply combining multiple scenarios during training introduces additional challenges, such as training instability and poor performance. To overcome these challenges, we propose Divide-Fuse-Conquer, a framework designed to enhance generalization in multi-scenario RL. This approach starts by heuristically grouping games based on characteristics such as rules and difficulties. Specialized models are then trained for each group to excel at games in the group is what we refer to as the divide step. Next, we fuse model parameters from different groups as a new model, and continue training it for multiple groups, until the scenarios in all groups are conquered. Experiments across 18 TextArena games show that Qwen2.5-32B-Align trained with the Divide-Fuse-Conquer strategy reaches a performance level comparable to Claude3.5, achieving 7 wins and 4 draws. We hope our approach can inspire future research on using reinforcement learning to improve the generalization of LLMs.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 23 of its 32 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

23 of 32 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning20 Feb
  2. 2025Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search4 Feb
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  4. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs22 Jan
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May

Other references · 18

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  3. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
  4. 2023Reward Design with Language Modelscited by 261
  5. 2025R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Modelcited by 165
  6. 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learningcited by 158
  7. 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 152
  8. 2025Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learningcited by 110
  9. 2023Level Generation Through Large Language Modelscited by 68
  10. 2025R1-Reward: Training Multimodal Reward Model Through Stable Reinforcement Learningcited by 65
  11. 2024CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agentscited by 42
  12. 2024Sample-Efficient Robust Multi-Agent Reinforcement Learning in the Face of Environmental Uncertaintycited by 25
6 more
  1. 2024Large Language Models and Video Games: A Preliminary Scoping Reviewcited by 23
  2. 2021Generalization in Text-based Games via Hierarchical Reinforcement Learningcited by 11
  3. 2019Rogue-Gym: A New Challenge for Generalization in Reinforcement Learningcited by 6
  4. 2025PCGRLLM: Large Language Model-Driven Reward Design for Procedural Content Generation Reinforcement Learningcited by 3
  5. 2025Reinforcement Learning in Strategy-Based and Atari Games: A Review of Google DeepMinds Innovationscited by 2
  6. 2025Grammar and Gameplay-aligned RL for Game Description Generation with LLMscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learningcited by 100
  2. 2025StepHint: Multi-level Stepwise Hints Enhance Reinforcement Learning to Reasoncited by 39
  3. 2026From Pixels to Digital Agents: An Empirical Study on the Taxonomy and Technological Trends of Reinforcement Learning Environmentscited by 0
  4. 2026Entropy Pacing Policy Optimization for Multi-Task Agentic Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Training LLMs for Divide-and-Conquer Reasoning Elevates Test-Time Scalability
  2. 2026Co-RL: Unsupervised Reasoning Emerges from Diverse Cohort in Multi-agent RL
  3. 2025Emergent Hierarchical Reasoning in LLMs through Reinforcement Learning
  4. 2025Divide and Conquer: Grounding LLMs as Efficient Decision-Making Agents via Offline Hierarchical Reinforcement Learning
  5. 2026MindGames Arena Generalization Track: In2AI Solution with Delayed Per-Step Reward Attribution
  6. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle
  7. 2024YOLO-MARL: You Only LLM Once for Multi-Agent Reinforcement Learning
  8. 2024Reasoning, Memorization, and Fine-Tuning Language Models for Non-Cooperative Games