mlsanityData to 7 Oct 2026

Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning

arXiv 2302.02662 ↗First posted 6 Feb 2023cs.LGcited by 193

Not on the map. Why?

Cited by 54 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent works successfully leveraged Large Language Models' (LLM) abilities to capture abstract knowledge about world's physics to solve decision-making problems. Yet, the alignment between LLMs' knowledge and the environment can be wrong and limit functional competence due to lack of grounding. In this paper, we study an approach (named GLAM) to achieve this alignment through functional grounding: we consider an agent using an LLM as a policy that is progressively updated as the agent interacts with the environment, leveraging online Reinforcement Learning to improve its performance to solve goals. Using an interactive textual environment designed to study higher-level forms of functional grounding, and a set of spatial and navigation tasks, we study several scientific questions: 1) Can LLMs boost sample efficiency for online learning of various RL tasks? 2) How can it boost different forms of generalization? 3) What is the impact of online learning? We study these questions by functionally grounding several variants (size, architecture) of FLAN-T5.

Key facts. Cited by 193 papers in the map (data through 7 Oct 2026). Builds on 46 of its 63 references that are in the map. First posted on arXiv 6 Feb 2023.

Builds on

46 of 63 references.

References · 46

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2017Proximal Policy Optimization Algorithmscited by 14,441
  7. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  8. 2020Scaling Laws for Neural Language Modelscited by 6,411
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  12. 2020Learning to summarize from human feedbackcited by 2,562
34 more
  1. 2022Emergent Abilities of Large Language Modelscited by 2,483
  2. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  3. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  4. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  5. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  6. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  7. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  8. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  9. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  10. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  11. 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 420
  12. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  13. 2018TextWorld: A Learning Environment for Text-based Gamescited by 304
  14. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 298
  15. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 244
  16. 2020Experience Grounds Languagecited by 241
  17. 2018BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learningcited by 238
  18. 2022Pre-Trained Language Models for Interactive Decision-Makingcited by 220
  19. 2022Online Decision Transformercited by 194
  20. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  21. 2021Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Colorcited by 118
  22. 2017Grounded Language Learning in a Simulated 3D Worldcited by 100
  23. 2017Mapping Instructions and Visual Observations to Actions with Reinforcement Learningcited by 80
  24. 2018Learning to Understand Goal Specifications by Modelling Rewardcited by 80
  25. 2022Can Wikipedia Help Offline Reinforcement Learning?cited by 76
  26. 2020Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Explorationcited by 68
  27. 2023Collaborating with language models for embodied reasoningcited by 61
  28. 2020Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Textcited by 59
  29. 2019Distributional Semantics and Linguistic Theorycited by 53
  30. 2021AW-Opt: Learning Robotic Skills with Imitation and Reinforcement at Scalecited by 43
  31. 2021ELLA: Exploration through Learned Language Abstractioncited by 42
  32. 2021A Systematic Survey of Text Worlds as Embodied Natural Language Environmentscited by 17
  33. 2022Foundation Models for Semantic Novelty in Reinforcement Learningcited by 9
  34. 2022On the Effect of Pre-training for Transformer in Different Modality on Offline Reinforcement Learningcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 46; 2024: 63; 2025: 56; 2026: 28.

193 papers.

Sort:

By area:

  1. 2023A Survey of Large Language Modelscited by 2,537
  2. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
  3. 2023Augmented Language Models: a Surveycited by 398
  4. 2023Language Models can Solve Computer Taskscited by 371
  5. 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 253
  6. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  7. 2024Teaching Large Language Models to Reason with Reinforcement Learningcited by 158
  8. 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learningcited by 158
  9. 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learningcited by 153
  10. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learningcited by 146
  11. 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 141
  12. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policycited by 139
38 more
  1. 2023Language Models Meet World Models: Embodied Experiences Enhance Language Modelscited by 123
  2. 2024DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoningcited by 119
  3. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 114
  4. 2025Improving Vision-Language-Action Model with Online Reinforcement Learningcited by 107
  5. 2024A Survey on Large Language Model-Based Game Agentscited by 106
  6. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 99
  7. 2024OpenR: An Open Source Framework for Advanced Reasoning with Large Language Modelscited by 92
  8. 2023Large Language Models as Generalizable Policies for Embodied Taskscited by 86
  9. 2024o1-Coder: an o1 Replication for Codingcited by 77
  10. 2023Everything of Thoughts: Defying the Law of Penrose Triangle for Thought Generationcited by 73
  11. 2025SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experiencecited by 68
  12. 2023Learning to Model the World with Languagecited by 66
  13. 2023Large Language Models in Education: Vision and Opportunitiescited by 54
  14. 2024SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priorscited by 53
  15. 2023Large Generative AI Models for Telecom: The Next Big Thing?cited by 52
  16. 2025MARFT: Multi-Agent Reinforcement Fine-Tuningcited by 51
  17. 2023Tree-Planner: Efficient Close-loop Task Planning with Large Language Modelscited by 48
  18. 2025Rewarding Doubt: A Reinforcement Learning Approach to Calibrated Confidence Expression of Large Language Modelscited by 48
  19. 2023Large Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergencecited by 42
  20. 2024On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS)cited by 40
  21. 2023Zero-Shot Goal-Directed Dialogue via RL on Imagined Conversationscited by 38
  22. 2023Visual cognition in multimodal large language modelscited by 38
  23. 2024True Knowledge Comes from Practice: Aligning LLMs with Embodied Environments via Reinforcement Learningcited by 36
  24. 2024Grounding Multimodal Large Language Models in Actionscited by 35
  25. 2024RL-GPT: Integrating Reinforcement Learning and Code-as-policycited by 32
  26. 2024Intelligent Go-Explore: Standing on the Shoulders of Giant Foundation Modelscited by 32
  27. 2023Interactive Natural Language Processingcited by 31
  28. 2023Large Language Model as a Policy Teacher for Training Reinforcement Learning Agentscited by 30
  29. 2024Efficient Reinforcement Learning with Large Language Model Priorscited by 30
  30. 2023Augmenting Autotelic Agents with Large Language Modelscited by 29
  31. 2023Enabling Intelligent Interactions between an Agent and an LLM: A Reinforcement Learning Approachcited by 28
  32. 2024LLM-Empowered State Representation for Reinforcement Learningcited by 28
  33. 2023Emptying the Ocean with a Spoon: Should We Edit Models?cited by 27
  34. 2024Large Multi-Modal Models (LMMs) as Universal Foundation Models for AI-Native Wireless Systemscited by 27
  35. 2025Multi-Turn Code Generation Through Single-Step Rewardscited by 27
  36. 2025World Modeling Makes a Better Planner: Dual Preference Optimization for Embodied Task Planningcited by 26
  37. 2023Are Large Language Models Temporally Grounded?cited by 25
  38. 2023The SocialAI School: Insights from Developmental Psychology Towards Artificial Socio-Cultural Agentscited by 24

50 most cited of 193 All 193 (JSON)