mlsanityData to 14 Oct 2026

RLAP: A Reinforcement Learning Enhanced Adaptive Planning Framework for Multi-step NLP Task Solving

arXiv 2505.11893 ↗First posted 17 May 2025cs.CL cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

Multi-step planning has been widely employed to enhance the performance of large language models (LLMs) on downstream natural language processing (NLP) tasks, which decomposes the original task into multiple subtasks and guide LLMs to solve them sequentially without additional training. When addressing task instances, existing methods either preset the order of steps or attempt multiple paths at each step. However, these methods overlook instances' linguistic features and rely on the intrinsic planning capabilities of LLMs to evaluate intermediate feedback and then select subtasks, resulting in suboptimal outcomes. To better solve multi-step NLP tasks with LLMs, in this paper we propose a Reinforcement Learning enhanced Adaptive Planning framework (RLAP). In our framework, we model an NLP task as a Markov decision process (MDP) and employ an LLM directly into the environment. In particular, a lightweight Actor model is trained to estimate Q-values for natural language sequences consisting of states and actions through reinforcement learning. Therefore, during sequential planning, the linguistic features of each sequence in the MDP can be taken into account, and the Actor model interacts with the LLM to determine the optimal order of subtasks for each task instance. We apply RLAP on three different types of NLP tasks and conduct extensive experiments on multiple datasets to verify RLAP's effectiveness and robustness.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 31 of its 48 references that are in the map. First posted on arXiv 17 May 2025.

Builds on

31 of 48 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2020Learning to summarize from human feedbackcited by 2,588
  5. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  6. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  7. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  8. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 861
  9. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  10. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 501
  11. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 392
  12. 2016Deep Reinforcement Learning for Dialogue Generationcited by 288
19 more
  1. 2023ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPTcited by 250
  2. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 236
  3. 2024When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Methodcited by 204
  4. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  5. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 164
  6. 2023Task and Motion Planning with Large Language Models for Object Rearrangementcited by 141
  7. 2018A Span-Extraction Dataset for Chinese Machine Reading Comprehensioncited by 84
  8. 2016Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Criticcited by 81
  9. 2019Investigating Prior Knowledge for Challenging Chinese Machine Reading Comprehensioncited by 81
  10. 2018Reliability and Learnability of Human Bandit Feedback for Sequence-to-Sequence Reinforcement Learningcited by 57
  11. 2021Reordering Examples Helps during Priming-based Few-Shot Learningcited by 50
  12. 2023Learning to Generate Better Than Your LLMcited by 49
  13. 2018Robust Distant Supervision Relation Extraction via Deep Reinforcement Learningcited by 35
  14. 2021Revisiting the Negative Data of Distantly Supervised Relation Extractioncited by 15
  15. 2020A Sentence Cloze Dataset for Chinese Machine Reading Comprehensioncited by 14
  16. 2018Reinforced Co-Trainingcited by 7
  17. 2018Deep Reinforcement Learning for Chinese Zero pronoun Resolutioncited by 5
  18. 2024Improving Recall of Large Language Models: A Model Collaboration Approach for Relational Triple Extractioncited by 3
  19. 2024Adaptive Reinforcement Learning Planning: Harnessing Large Language Models for Complex Information Extractioncited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods
  2. 2024Reinforcement Learning Problem Solving with Large Language Models
  3. 2025Benefits and Pitfalls of Reinforcement Learning for Language Model Planning: A Theoretical Perspective
  4. 2024How Can LLM Guide RL? A Value-Based Approach
  5. 2025A Technical Survey of Reinforcement Learning Techniques for Large Language Models
  6. 2025Feedback-Induced Performance Decline in LLM-Based Decision-Making
  7. 2024The RL/LLM Taxonomy Tree: Reviewing Synergies Between Reinforcement Learning and Large Language Models
  8. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle