mlsanityData to 7 Oct 2026

Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learning

arXiv 2101.07393 ↗First posted 19 Jan 2021cs.CL cs.AI cs.LGcited by 45

Not on the map. Why?

Abstract (arXiv)

We investigate the use of natural language to drive the generalization of control policies and introduce the new multi-task environment Messenger with free-form text manuals describing the environment dynamics. Unlike previous work, Messenger does not assume prior knowledge connecting text and state observations - the control policy must simultaneously ground the game manual to entity symbols and dynamics in the environment. We develop a new model, EMMA (Entity Mapper with Multi-modal Attention) which uses an entity-conditioned attention module that allows for selective focus over relevant descriptions in the manual for each entity in the environment. EMMA is end-to-end differentiable and learns a latent grounding of entities and dynamics from text to observations using only environment rewards. EMMA achieves successful zero-shot generalization to unseen games with new dynamics, obtaining a 40% higher win rate compared to multiple baselines. However, win rate on the hardest stage of Messenger remains low (10%), demonstrating the need for additional work in this direction.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Builds on 23 of its 51 references that are in the map. First posted on arXiv 19 Jan 2021.

Builds on

23 of 51 references.

References · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2017Proximal Policy Optimization Algorithmscited by 14,441
  5. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  6. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  7. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 200
  8. 2020The NetHack Learning Environmentcited by 196
  9. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  10. 2017Grounded Language Learning in a Simulated 3D Worldcited by 100
  11. 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 97
  12. 2017Mapping Instructions and Visual Observations to Actions with Reinforcement Learningcited by 80
11 more
  1. 2020Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Textcited by 59
  2. 2017Unified Pragmatic Models for Generating and Following Instructionscited by 55
  3. 2019Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigationcited by 54
  4. 2020Grounded Language Learning Fast and Slowcited by 54
  5. 2019Environmental drivers of systematicity and generalization in a situated agentcited by 50
  6. 2019Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flightcited by 43
  7. 2014Learning to Win by Reading Manuals in a Monte-Carlo Frameworkcited by 39
  8. 2018Guiding Policies with Language via Meta-Learningcited by 34
  9. 2017Representation Learning for Grounded Spatial Reasoningcited by 23
  10. 2015Alignment-based compositional semantics for instruction followingcited by 21
  11. 2016Navigational Instruction Generation as Inverse Reinforcement Learning with Neural Machine Translationcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 11; 2023: 18; 2024: 8; 2025: 4; 2026: 1.

45 papers.

Sort:

By area:

  1. 2023Cognitive Architectures for Language Agentscited by 422
  2. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 257
  3. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  4. 2023SmartPlay: A Benchmark for LLMs as Intelligent Agentscited by 123
  5. 2025Improving Vision-Language-Action Model with Online Reinforcement Learningcited by 107
  6. 2023ProAgent: Building Proactive Cooperative Agents with Large Language Modelscited by 97
  7. 2023Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modellingcited by 88
  8. 2024AutoGuide: Automated Generation and Selection of Context-Aware Guidelines for Large Language Model Agentscited by 69
  9. 2023Learning to Model the World with Languagecited by 66
  10. 2023XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAXcited by 54
  11. 2022Improving Intrinsic Exploration with Language Abstractionscited by 49
  12. 2023Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manualscited by 45
33 more
  1. 2023LLF-Bench: Benchmark for Interactive Learning from Language Feedbackcited by 30
  2. 2022Leveraging Language for Accelerated Learning of Tool Manipulationcited by 26
  3. 2022Law Informs Code: A Legal Informatics Approach to Aligning Artificial Intelligence with Humanscited by 23
  4. 2022How to talk so AI will learn: Instructions, descriptions, and autonomycited by 22
  5. 2022Language Control Diffusion: Efficiently Scaling through Space, Time, and Taskscited by 20
  6. 2024Language Grounded Multi-agent Reinforcement Learning with Human-interpretable Communicationcited by 17
  7. 2021Zero-shot Task Adaptation using Natural Languagecited by 15
  8. 2024Language-Guided World Models: A Model-Based Approach to AI Controlcited by 12
  9. 2023SPRING: Studying the Paper and Reasoning to Play Gamescited by 11
  10. 2021SILG: The Multi-environment Symbolic Interactive Language Grounding Benchmarkcited by 10
  11. 2023AdaRefiner: Refining Decisions of Language Models with Adaptive Feedbackcited by 10
  12. 2022Improving Policy Learning via Language Dynamics Distillationcited by 9
  13. 2023Large Language Models as Fiduciaries: A Case Study Toward Robustly Communicating With Artificial Intelligence Through Legal Standardscited by 8
  14. 2023Conceptual Reinforcement Learning for Language-Conditioned Taskscited by 8
  15. 2023Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learningcited by 8
  16. 2023Improving Long-Horizon Imitation Through Instruction Predictioncited by 8
  17. 2024Teaching Embodied Reinforcement Learning Agents: Informativeness and Diversity of Language Usecited by 7
  18. 2023Reinforcement Learning with Knowledge Representation and Reasoning: A Brief Surveycited by 5
  19. 2022Compositional Generalization in Grounded Language Learning via Induced Model Sparsitycited by 4
  20. 2023FoMo Rewards: Can we cast foundation models as reward functions?cited by 4
  21. 2022Learning to Query Internet Text for Informing Reinforcement Learning Agentscited by 3
  22. 2022lilGym: Natural Language Visual Reasoning with Reinforcement Learningcited by 3
  23. 2022SemSup: Semantic Supervision for Simple and Scalable Zero-shot Generalizationcited by 2
  24. 2022Entity Divider with Language Grounding in Multi-Agent Reinforcement Learningcited by 2
  25. 2023Language-guided Task Adaptation for Imitation Learningcited by 2
  26. 2025CrafText Benchmark: Advancing Instruction Following in Complex Multimodal Open-Ended Worldcited by 2
  27. 2023Improve the efficiency of deep reinforcement learning through semantic exploration guided by natural languagecited by 1
  28. 2021Feudal Reinforcement Learning by Reading Manualscited by 0
  29. 2024Instruction Following with Goal-Conditioned Reinforcement Learning in Virtual Environmentscited by 0
  30. 2024CAREL: Instruction-guided reinforcement learning with cross-modal auxiliary objectivescited by 0
  31. 2025Inductive Biases for Zero-shot Systematic Generalization in Language-informed Reinforcement Learningcited by 0
  32. 2025FOSSIL: Harnessing Feedback on Suboptimal Samples for Data-Efficient Generalisation with Imitation Learning for Embodied Vision-and-Language Taskscited by 0
  33. 2026LLM-Guided Contextual Action Evaluation for Operational Decisions in Industrial Processescited by 0