mlsanityData to 14 Oct 2026

Integrating Large Language Models and Reinforcement Learning for Non-Linear Reasoning

arXiv 2410.13501 ↗First posted 17 Oct 2024cs.LG cs.PLcited by 1

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Large Language Models (LLMs) were shown to struggle with long-term planning, which may be caused by the limited way in which they explore the space of possible solutions. We propose an architecture where a Reinforcement Learning (RL) Agent guides an LLM's space exploration: (1) the Agent has access to domain-specific information, and can therefore make decisions about the quality of candidate solutions based on specific and relevant metrics, which were not explicitly considered by the LLM's training objective; (2) the LLM can focus on generating immediate next steps, without the need for long-term planning. We allow non-linear reasoning by exploring alternative paths and backtracking. We evaluate this architecture on the program equivalence task, and compare it against Chain of Thought (CoT) and Tree of Thoughts (ToT). We assess both the downstream task, denoting the binary classification, and the intermediate reasoning steps. Our approach compares positively against CoT and ToT.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 18 of its 43 references that are in the map. First posted on arXiv 17 Oct 2024.

Builds on

18 of 43 references.

References · 18

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2017Graph Attention Networkscited by 8,923
  4. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  5. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 501
  6. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
  7. 2022FOLIO: Natural Language Reasoning with First-Order Logiccited by 235
  8. 2023Large Language Model Guided Tree-of-Thoughtcited by 221
  9. 2022MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoningcited by 156
  10. 2023Learning to Generate Better Than Your LLMcited by 49
  11. 2024CodeMind: Evaluating Large Language Models for Code Reasoningcited by 32
  12. 2023Towards Understanding the Capability of Large Language Models on Code Clone Detection: A Surveycited by 15
6 more
  1. 2023A systematic literature review on source code similarity measurement and clone detection: techniques, applications, and challengescited by 14
  2. 2023Large Language Model Programscited by 11
  3. 2024Evaluating the Effectiveness of GPT-4 Turbo in Creating Defeaters for Assurance Casescited by 6
  4. 2023An empirical study of ChatGPT-3.5 on question answering and code maintenancecited by 4
  5. 2020Equivalence of Dataflow Graphs via Rewrite Rules Using a Graph-to-Sequence Neural Modelcited by 3
  6. 2021Proving Equivalence Between Complex Expressions Using Graph-to-Sequence Neural Modelscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026CodePivot: Bootstrapping Multilingual Transpilation in LLMs via Reinforcement Learning without Parallel Corporacited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey of Reinforcement Learning for Large Reasoning Models
  2. 2025Interleaved Reasoning for Large Language Models via Reinforcement Learning
  3. 2025Agent-as-Tool: A Study on the Hierarchical Decision Making with Reinforcement Learning
  4. 2025Large Language and Reasoning Models are Shallow Disjunctive Reasoners
  5. 2026The Art of Efficient Reasoning: Data, Reward, and Optimization
  6. 2025Reasoning Under 1 Billion: Memory-Augmented Reinforcement Learning for Large Language Models
  7. 2025Demystifying Long Chain-of-Thought Reasoning in LLMs
  8. 2025Not All Thoughts are Generated Equal: Efficient LLM Reasoning via Multi-Turn Reinforcement Learning