mlsanityData to 7 Oct 2026

Language Guided Exploration for RL Agents in Text Environments

arXiv 2403.03141 ↗First posted 5 Mar 2024cs.CLcited by 9

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Real-world sequential decision making is characterized by sparse rewards and large decision spaces, posing significant difficulty for experiential learning systems like textittabula rasa reinforcement learning (RL) agents. Large Language Models (LLMs), with a wealth of world knowledge, can help RL agents learn quickly and adapt to distribution shifts. In this work, we introduce Language Guided Exploration (LGE) framework, which uses a pre-trained language model (called GUIDE ) to provide decision-level guidance to an RL agent (called EXPLORER). We observe that on ScienceWorld (Wang et al.,2022), a challenging text environment, LGE outperforms vanilla RL agents significantly and also outperforms other sophisticated methods like Behaviour Cloning and Text Decision Transformer.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 23 of its 28 references that are in the map. First posted on arXiv 5 Mar 2024.

Builds on

23 of 28 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  3. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  4. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  5. 2018TextWorld: A Learning Environment for Text-based Gamescited by 304
  6. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 298
  7. 2022Pre-Trained Language Models for Interactive Decision-Makingcited by 220
  8. 2019Interactive Fiction Games: A Colossal Adventurecited by 165
  9. 2020Keep CALM and Explore: Language Models for Action Generation in Text-based Gamescited by 108
  10. 2019Procedure Planning in Instructional Videoscited by 92
  11. 2020Graph Constrained Reinforcement Learning for Natural Language Action Spacescited by 80
  12. 2019A survey on intrinsic motivation in reinforcement learningcited by 79
11 more
  1. 2018Learn What Not to Learn: Action Elimination with Deep Reinforcement Learningcited by 77
  2. 2021What Would Jiminy Cricket Do? Towards Agents That Behave Morallycited by 57
  3. 2023Language Reward Modulation for Pretraining Reinforcement Learningcited by 42
  4. 2021General-Purpose Question-Answering with Macawcited by 40
  5. 2020Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Baselinescited by 37
  6. 2023Pretrained Language Models as Visual Planners for Human Assistancecited by 28
  7. 2024Policy Improvement using Language Feedback Modelscited by 16
  8. 2022Behavior Cloned Transformers are Neurosymbolic Reasonerscited by 10
  9. 2021Case-based Reasoning for Better Generalization in Textual Reinforcement Learningcited by 8
  10. 2023Intrinsic Language-Guided Exploration for Complex Long-Horizon Robotic Manipulation Taskscited by 8
  11. 2019Learn How to Cook a New Recipe in a New House: Using Map Familiarization, Curriculum Learning, and Bandit Feedback to Learn Families of Text-Based Adventure Gamescited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 1.

9 papers.

  1. 2024A Survey on Large Language Model-Based Game Agentscited by 106
  2. 2025SAMULE: Self-Learning Agents Enhanced by Multi-level Reflectioncited by 15
  3. 2025TALES: Text Adventure Learning Environment Suitecited by 9
  4. 2025Fox in the Henhouse: Supply-Chain Backdoor Attacks Against Reinforcement Learningcited by 5
  5. 2026InterPReT: Interactive Policy Restructuring and Training Enable Effective Imitation Learning from Laypersonscited by 2
  6. 2025Attacks and Defenses Against LLM Fingerprintingcited by 1
  7. 2025AutoContext: Instance-Level Context Learning for LLM Agentscited by 1
  8. 2025InstructVTON: Optimal Auto-Masking and Natural-Language-Guided Interactive Style Control for Inpainting-Based Virtual Try-Oncited by 0
  9. 2025ByteSized32Refactored: Towards an Extensible Interactive Text Games Corpus for LLM World Modeling and Evaluationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Toward Efficient Exploration by Large Language Model Agents
  2. 2024Words as Beacons: Guiding RL Agents with High-Level Language Prompts
  3. 2025Guiding Exploration in Reinforcement Learning Through LLM-Augmented Observations
  4. 2026Expanding LLM Agent Boundaries with Strategy-Guided Exploration
  5. 2025Meta-RL Induces Exploration in Language Agents
  6. 2025Representation-Based Exploration for Language Models: From Test-Time to Post-Training
  7. 2023Towards A Unified Agent with Foundation Models
  8. 2025LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Models