# RTFM: Generalising to Novel Environment Dynamics via Reading

arXiv: https://arxiv.org/abs/1910.08210 · HTML: https://mlsanity.com/paper/1910.08210/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Oct 2019.

## Built on by (most cited first, 30 of 33)

- [Language Conditioned Imitation Learning over Unstructured Data](https://mlsanity.com/paper/2005.07648/) (2020)
- [A Survey of Zero-shot Generalisation in Deep Reinforcement Learning](https://mlsanity.com/paper/2111.09794/) (2021)
- [DocPrompting: Generating Code by Retrieving the Docs](https://mlsanity.com/paper/2207.05987/) (2022)
- [Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation](https://mlsanity.com/paper/2308.07931/) (2023)
- [SmartPlay: A Benchmark for LLMs as Intelligent Agents](https://mlsanity.com/paper/2310.01557/) (2023)
- [Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modelling](https://mlsanity.com/paper/2301.12050/) (2023)
- [Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models](https://mlsanity.com/paper/2308.00675/) (2023)
- [Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manuals](https://mlsanity.com/paper/2302.04449/) (2023)
- [Distilling Internet-Scale Vision-Language Models into Embodied Agents](https://mlsanity.com/paper/2301.12507/) (2023)
- [Leveraging Language for Accelerated Learning of Tool Manipulation](https://mlsanity.com/paper/2206.13074/) (2022)
- [How to talk so AI will learn: Instructions, descriptions, and autonomy](https://mlsanity.com/paper/2206.07870/) (2022)
- [Language Control Diffusion: Efficiently Scaling through Space, Time, and Tasks](https://mlsanity.com/paper/2210.15629/) (2022)
- [Policy Improvement using Language Feedback Models](https://mlsanity.com/paper/2402.07876/) (2024)
- [G^3: Geolocation via Guidebook Grounding](https://mlsanity.com/paper/2211.15521/) (2022)
- [SILG: The Multi-environment Symbolic Interactive Language Grounding Benchmark](https://mlsanity.com/paper/2110.10661/) (2021)
- [Asking for Knowledge: Training RL Agents to Query External Knowledge Using Language](https://mlsanity.com/paper/2205.06111/) (2022)
- [Conceptual Reinforcement Learning for Language-Conditioned Tasks](https://mlsanity.com/paper/2303.05069/) (2023)
- [Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2305.10865/) (2023)
- [RLang: A Declarative Language for Describing Partial World Knowledge to Reinforcement Learning Agents](https://mlsanity.com/paper/2208.06448/) (2022)
- [Learning to Query Internet Text for Informing Reinforcement Learning Agents](https://mlsanity.com/paper/2205.13079/) (2022)
- [Grid-to-Graph: Flexible Spatial Relational Inductive Biases for Reinforcement Learning](https://mlsanity.com/paper/2102.04220/) (2021)
- [Entity Divider with Language Grounding in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2210.13942/) (2022)
- [CrafText Benchmark: Advancing Instruction Following in Complex Multimodal Open-Ended World](https://mlsanity.com/paper/2505.11962/) (2025)
- [Ask Before You Act: Generalising to Novel Environments by Asking Questions](https://mlsanity.com/paper/2209.04665/) (2022)
- [Heterogeneous Knowledge for Augmented Modular Reinforcement Learning](https://mlsanity.com/paper/2306.01158/) (2023)
- [A Data Source for Reasoning Embodied Agents](https://mlsanity.com/paper/2309.07974/) (2023)
- [Improve the efficiency of deep reinforcement learning through semantic exploration guided by natural language](https://mlsanity.com/paper/2309.11753/) (2023)
- [PSALM-V: Automating Symbolic Planning in Interactive Visual Environments with Large Language Models](https://mlsanity.com/paper/2506.20097/) (2025)
- [Procedures as Programs: Hierarchical Control of Situated Agents through Natural Language](https://mlsanity.com/paper/2109.08214/) (2021)
- [Learning Invariable Semantical Representation from Language for Extensible Policy Generalization](https://mlsanity.com/paper/2202.00466/) (2022)

## Abstract (arXiv)

Obtaining policies that can generalise to new environments in reinforcement learning is challenging. In this work, we demonstrate that language understanding via a reading policy learner is a promising vehicle for generalisation to new environments. We propose a grounded policy learning problem, Read to Fight Monsters (RTFM), in which the agent must jointly reason over a language goal, relevant dynamics described in a document, and environment observations. We procedurally generate environment dynamics and corresponding language descriptions of the dynamics, such that agents must read to understand new environment dynamics instead of memorising any particular information. In addition, we propose txt2π, a model that captures three-way interactions between the goal, document, and observations. On RTFM, txt2π generalises to new environments with dynamics not seen during training via reading. Furthermore, our model outperforms baselines such as FiLM and language-conditioned CNNs on RTFM. Through curriculum learning, txt2π produces policies that excel on complex RTFM tasks requiring several reasoning and coreference steps.