mlsanityData to 14 Oct 2026

Divide and Conquer: Grounding LLMs as Efficient Decision-Making Agents via Offline Hierarchical Reinforcement Learning

arXiv 2505.19761 ↗First posted 26 May 2025cs.AIcited by 2

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 2 more
Abstract (arXiv)

While showing sophisticated reasoning abilities, large language models (LLMs) still struggle with long-horizon decision-making tasks due to deficient exploration and long-term credit assignment, especially in sparse-reward scenarios. Inspired by the divide-and-conquer principle, we propose an innovative framework **GLIDER** (**G**rounding **L**anguage Models as Eff**I**cient **D**ecision-Making Agents via Offline Hi**E**rarchical **R**einforcement Learning) that introduces a parameter-efficient and generally applicable hierarchy to LLM policies. We develop a scheme where the low-level controller is supervised with abstract, step-by-step plans that are learned and instructed by the high-level policy. This design decomposes complicated problems into a series of coherent chain-of-thought reasoning sub-tasks, providing flexible temporal abstraction to significantly enhance exploration and learning for long-horizon tasks. Furthermore, GLIDER facilitates fast online adaptation to non-stationary environments owing to the strong transferability of its task-agnostic low-level skills. Experiments on ScienceWorld and ALFWorld benchmarks show that GLIDER achieves consistent performance gains, along with enhanced generalization capabilities.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 12 of its 63 references that are in the map. First posted on arXiv 26 May 2025.

Builds on

12 of 63 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan

Other references · 11

  1. 2023Mistral 7Bcited by 3,164
  2. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  4. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  5. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  6. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 916
  7. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 671
  8. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 338
  9. 2023FireAct: Toward Language Agent Fine-tuningcited by 256
  10. 2023Strategic Reasoning with Language Modelscited by 64
  11. 2024Learning From Failure: Integrating Negative Examples when Fine-tuning Large Language Models as Agentscited by 50

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  2. 2026Hierarchical Chain-of-Thought: Enhancing LLM Reasoning Performance and Efficiencycited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Training LLMs for Divide-and-Conquer Reasoning Elevates Test-Time Scalability
  2. 2025Emergent Hierarchical Reasoning in LLMs through Reinforcement Learning
  3. 2026HiMAC: Hierarchical Macro-Micro Learning for Long-Horizon LLM Agents
  4. 2024Integrating Large Language Models and Reinforcement Learning for Non-Linear Reasoning
  5. 2025Can LLMs Guide Their Own Exploration? Gradient-Guided Reinforcement Learning for LLM Reasoning
  6. 2026A Subgoal-driven Framework for Improving Long-Horizon LLM Agents
  7. 2026Hierarchical Control in Multi-Agent Games: LLM-based Planning and RL Execution
  8. 2024From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems