mlsanityData to 7 Oct 2026

Pre-Trained Language Models for Interactive Decision-Making

arXiv 2202.01771 ↗First posted 3 Feb 2022cs.LG cs.CLcited by 220

Not on the map. Why?

Cited by 39 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Language model (LM) pre-training is useful in many language processing tasks. But can pre-trained LMs be further leveraged for more general machine learning problems? We propose an approach for using LMs to scaffold learning and generalization in general sequential decision-making problems. In this approach, goals and observations are represented as a sequence of embeddings, and a policy network initialized with a pre-trained LM predicts the next action. We demonstrate that this framework enables effective combinatorial generalization across different environments and supervisory modalities. We begin by assuming access to a set of expert demonstrations, and show that initializing policies with LMs and fine-tuning them via behavior cloning improves task completion rates by 43.6% in the VirtualHome environment. Next, we integrate an active data gathering procedure in which agents iteratively interact with the environment, relabel past "failed" experiences with new goals, and update their policies in a self-supervised loop. Active data gathering further improves combinatorial generalization, outperforming the best baseline by 25.1%. Finally, we explain these results by investigating three possible factors underlying the effectiveness of the LM-based policy. We find that sequential input representations (vs. fixed-dimensional feature vectors) and LM-based weight initialization are both important for generalization. Surprisingly, however, the format of the policy inputs encoding (e.g. as a natural language string vs. an arbitrary sequential encoding) has little influence. Together, these results suggest that language modeling induces representations that are useful for modeling not just language, but also goals and plans; these representations can aid learning and generalization even outside of language processing.

Key facts. Cited by 220 papers in the map (data through 7 Oct 2026). Builds on 47 of its 74 references that are in the map. First posted on arXiv 3 Feb 2022.

Builds on

47 of 74 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2017Proximal Policy Optimization Algorithmscited by 14,441
  6. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  7. 2015Continuous control with deep reinforcement learningcited by 3,279
  8. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  9. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  10. 2018Deep contextualized word representationscited by 2,285
  11. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  12. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
35 more
  1. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  2. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  3. 2017Hindsight Experience Replaycited by 1,096
  4. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  5. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  6. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  7. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  8. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
  9. 2018Behavioral Cloning from Observationcited by 482
  10. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
  11. 2018VirtualHome: Simulating Household Activities via Programscited by 411
  12. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  13. 2018TextWorld: A Learning Environment for Text-based Gamescited by 304
  14. 2019A Multiscale Visualization of Attention in the Transformer Modelcited by 292
  15. 2021Implicit Representations of Meaning in Neural Language Modelscited by 179
  16. 2020Improving Vision-and-Language Navigation with Image-Text Pairs from the Webcited by 177
  17. 2021Pretrained Transformers as Universal Computation Enginescited by 162
  18. 2020Watch-And-Help: A Challenge for Social Perception and Human-AI Collaborationcited by 131
  19. 2021Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Colorcited by 118
  20. 2018Multilingual Constituency Parsing with Self-Attention and Pre-Trainingcited by 117
  21. 2020Keep CALM and Explore: Language Models for Action Generation in Text-based Gamescited by 108
  22. 2018Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flowcited by 103
  23. 2021Skill Induction and Planning with Latent Languagecited by 99
  24. 2022Can Wikipedia Help Offline Reinforcement Learning?cited by 76
  25. 2017Learning with Latent Languagecited by 75
  26. 2015Language Understanding for Text-based Games Using Deep Reinforcement Learningcited by 70
  27. 2020Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Textcited by 59
  28. 2021Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completioncited by 57
  29. 2018Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learningcited by 53
  30. 2021Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoringcited by 53
  31. 2018Counting to Explore and Generalize in Text-based Gamescited by 36
  32. 2021Reading and Acting while Blindfolded: The Need for Semantics in Text Game Agentscited by 20
  33. 2017RAIL: Risk-Averse Imitation Learningcited by 9
  34. 2020Probing Contextual Language Models for Common Ground with Visual Representationscited by 9
  35. 2021Multitasking Inhibits Semantic Driftcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 31; 2023: 69; 2024: 64; 2025: 38; 2026: 18.

220 papers.

Sort:

By area:

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  3. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  4. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  5. 2022A Generalist Agentcited by 802
  6. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  7. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  8. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  9. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  10. 2023Augmented Language Models: a Surveycited by 398
  11. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
  12. 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 302
38 more
  1. 2023Tool Learning with Foundation Modelscited by 302
  2. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  3. 2022The Alignment Problem from a Deep Learning Perspectivecited by 246
  4. 2023Can Large Language Models Transform Computational Social Science?cited by 245
  5. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 238
  6. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 231
  7. 2023Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planningcited by 204
  8. 2023Large Language Models as General Pattern Machinescited by 196
  9. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 193
  10. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 170
  11. 2022Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editingcited by 165
  12. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  13. 2023Video Language Planningcited by 160
  14. 2023Multimodal Web Navigation with Instruction-Finetuned Foundation Modelscited by 159
  15. 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 159
  16. 2022LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Taskscited by 146
  17. 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 141
  18. 2023Translating Natural Language to Planning Goals with Large-Language Modelscited by 132
  19. 2023Language Models Meet World Models: Embodied Experiences Enhance Language Modelscited by 123
  20. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  21. 2023Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Modelcited by 115
  22. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 114
  23. 2023Bayesian Low-rank Adaptation for Large Language Modelscited by 112
  24. 2024A Survey on Large Language Model-Based Game Agentscited by 106
  25. 2024Cognitive Bias in Decision-Making with LLMscited by 104
  26. 2025Multi-Agent Collaboration via Evolving Orchestrationcited by 102
  27. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 99
  28. 2025TesserAct: Learning 4D Embodied World Modelscited by 90
  29. 2024Large Language Models for Robotics: Opportunities, Challenges, and Perspectivescited by 88
  30. 2023Embodied Task Planning with Large Language Modelscited by 84
  31. 2024MMToM-QA: Multimodal Theory of Mind Question Answeringcited by 84
  32. 2022Mind's Eye: Grounded Language Model Reasoning through Simulationcited by 83
  33. 2025PlanGenLLMs: A Modern Survey of LLM Planning Capabilitiescited by 82
  34. 2023ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planningcited by 77
  35. 2023Making Language Models Better Tool Learners with Execution Feedbackcited by 69
  36. 2023Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agentscited by 66
  37. 2023Learning to Model the World with Languagecited by 66
  38. 2023LoRA ensembles for large language model fine-tuningcited by 65

50 most cited of 220 All 220 (JSON)