# Beating Atari with Natural Language Guided Reinforcement Learning

arXiv: https://arxiv.org/abs/1704.05539 · HTML: https://mlsanity.com/paper/1704.05539/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Apr 2017.

## Built on by (most cited first, 24 of 24)

- [Training Language Models with Language Feedback at Scale](https://mlsanity.com/paper/2303.16755/) (2023)
- [Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation](https://mlsanity.com/paper/2109.01115/) (2021)
- [Training Language Models with Language Feedback](https://mlsanity.com/paper/2204.14146/) (2022)
- [Learning Rewards from Linguistic Feedback](https://mlsanity.com/paper/2009.14715/) (2020)
- [Tell me why! Explanations support learning relational and causal structure](https://mlsanity.com/paper/2112.03753/) (2021)
- [Zero-shot Task Adaptation using Natural Language](https://mlsanity.com/paper/2106.02972/) (2021)
- [Transformers are Adaptable Task Planners](https://mlsanity.com/paper/2207.02442/) (2022)
- [A Survey On Enhancing Reinforcement Learning in Complex Environments: Insights from Human and LLM Feedback](https://mlsanity.com/paper/2411.13410/) (2024)
- [The Dark Side of Rich Rewards: Understanding and Mitigating Noise in VLM Rewards](https://mlsanity.com/paper/2409.15922/) (2024)
- [E-MAPP: Efficient Multi-Agent Reinforcement Learning with Parallel Program Guidance](https://mlsanity.com/paper/2212.02064/) (2022)
- [Dream to Adapt: Meta Reinforcement Learning by Latent Context Imagination and MDP Imagination](https://mlsanity.com/paper/2311.06673/) (2023)
- [SMLT: A Serverless Framework for Scalable and Adaptive Machine Learning Design and Training](https://mlsanity.com/paper/2205.01853/) (2022)
- [Safe Reinforcement Learning with Free-form Natural Language Constraints and Pre-Trained Language Models](https://mlsanity.com/paper/2401.07553/) (2024)
- [Reinforcement Learning with Physics-Informed Symbolic Program Priors for Zero-Shot Wireless Indoor Navigation](https://mlsanity.com/paper/2506.22365/) (2025)
- [Language-guided Task Adaptation for Imitation Learning](https://mlsanity.com/paper/2301.09770/) (2023)
- [A Reminder of its Brittleness: Language Reward Shaping May Hinder Learning for Instruction Following Agents](https://mlsanity.com/paper/2305.16621/) (2023)
- [DAIL: Beyond Task Ambiguity for Language-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2510.19562/) (2025)
- [Grounding Complex Navigational Instructions Using Scene Graphs](https://mlsanity.com/paper/2106.01607/) (2021)
- [Reinforcement Learning with Information-Theoretic Actuation](https://mlsanity.com/paper/2109.15147/) (2021)
- [Knowledge-Integrated Informed AI for National Security](https://mlsanity.com/paper/2202.03188/) (2022)
- [Effective Reward Specification in Deep Reinforcement Learning](https://mlsanity.com/paper/2412.07177/) (2024)
- [From Text to Trajectory: Exploring Complex Constraint Representation and Decomposition in Safe Reinforcement Learning](https://mlsanity.com/paper/2412.08920/) (2024)
- [HERAKLES: Hierarchical Skill Compilation for Open-ended LLM Agents](https://mlsanity.com/paper/2508.14751/) (2025)
- [Cooperative-Competitive Team Play of Real-World Craft Robots](https://mlsanity.com/paper/2602.21119/) (2026)

## Abstract (arXiv)

We introduce the first deep reinforcement learning agent that learns to beat Atari games with the aid of natural language instructions. The agent uses a multimodal embedding between environment observations and natural language to self-monitor progress through a list of English instructions, granting itself reward for completing instructions in addition to increasing the game score. Our agent significantly outperforms Deep Q-Networks (DQNs), Asynchronous Advantage Actor-Critic (A3C) agents, and the best agents posted to OpenAI Gym on what is often considered the hardest Atari 2600 environment: Montezuma's Revenge.