mlsanityData to 14 Oct 2026

Ask more, know better: Reinforce-Learned Prompt Questions for Decision Making with Large Language Models

arXiv 2310.18127 ↗First posted 27 Oct 2023cs.LG cs.AI cs.CLcited by 9

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Large language models (LLMs) demonstrate their promise in tackling complicated practical challenges by combining action-based policies with chain of thought (CoT) reasoning. Having high-quality prompts on hand, however, is vital to the framework's effectiveness. Currently, these prompts are handcrafted utilising extensive human labor, resulting in CoT policies that frequently fail to generalise. Human intervention is also required to develop grounding functions that ensure low-level controllers appropriately process CoT reasoning. In this paper, we propose a comprehensive training framework for complex task-solving, incorporating human prior knowledge into the learning of action policies. To that purpose, we offer a new leader-follower bilevel framework that is capable of learning to ask relevant questions (prompts) and subsequently undertaking reasoning to guide the learning of actions. The prompt policy is employed to make introspective revisions based on historical findings, leading the CoT process to consider the anticipated goals and generate outputs that lead to decisive, high-performing actions. The action policy subsequently learns to comprehend and integrate the CoT outputs to take actions. Our empirical data reveal that our framework outperforms leading methods in 5 decision-making tasks such as Overcooked and FourRoom.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 23 of its 49 references that are in the map. First posted on arXiv 27 Oct 2023.

Builds on

23 of 49 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 21

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  5. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  6. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  7. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  8. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  9. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  10. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  11. 2022PAL: Program-aided Language Modelscited by 652
  12. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 350
9 more
  1. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
  2. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 195
  3. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 176
  4. 2023ProAgent: Building Proactive Cooperative Agents with Large Language Modelscited by 98
  5. 2023Navigation with Large Language Models: Semantic Guesswork as a Heuristic for Planningcited by 89
  6. 2021Hierarchical Reinforcement Learning By Discovering Intrinsic Optionscited by 58
  7. 2023Joint Prompt Optimization of Stacked LLMs using Variational Inferencecited by 32
  8. 2023Introspective Tips: Large Language Model for In-Context Decision Makingcited by 28
  9. 2018Adaptive Decision Making via Entropy Minimizationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 4; 2026: 2.

9 papers.

  1. 2025ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learningcited by 79
  2. 2024Efficient Reinforcement Learning with Large Language Model Priorscited by 30
  3. 2024Privacy-preserved LLM Cascade via CoT-enhanced Policy Learningcited by 8
  4. 2025DSADF: Thinking Fast and Slow for Decision Makingcited by 7
  5. 2024Reinforcement Learning for Aligning Large Language Models Agents with Interactive Environments: Quantifying and Mitigating Prompt Overfittingcited by 4
  6. 2025HCRMP: A LLM-Hinted Contextual Reinforcement Learning Framework for Autonomous Drivingcited by 4
  7. 2025Agentic Episodic Controlcited by 1
  8. 2026On the Limits of Prompt-Conditioned Language Models as General-Purpose Learnerscited by 0
  9. 2026Policy-as-Skill: Governed LLM Decision Support with Evidence, Deterministic Control, and Auditcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
  2. 2024ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting
  3. 2023Generalizable Chain-of-Thought Prompting in Mixed-task Scenarios with Large Language Models
  4. 2026Where Do CoT Training Gains Land in LLM based Agents?
  5. 2023An automatically discovered chain-of-thought prompt generalizes to novel models and datasets
  6. 2023Active Prompting with Chain-of-Thought for Large Language Models
  7. 2025Why Prompt Design Matters and Works: A Complexity Analysis of Prompt Search Space in LLMs
  8. 2025Post-Training Large Language Models via Reinforcement Learning from Self-Feedback