mlsanityData to 7 Oct 2026

Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manuals

arXiv 2302.04449 ↗First posted 9 Feb 2023cs.LG cs.AI cs.CLcited by 45

Not on the map. Why?

Abstract (arXiv)

High sample complexity has long been a challenge for RL. On the other hand, humans learn to perform tasks not only from interaction or demonstrations, but also by reading unstructured text documents, e.g., instruction manuals. Instruction manuals and wiki pages are among the most abundant data that could inform agents of valuable features and policies or task-specific environmental dynamics and reward structures. Therefore, we hypothesize that the ability to utilize human-written instruction manuals to assist learning policies for specific tasks should lead to a more efficient and better-performing agent. We propose the Read and Reward framework. Read and Reward speeds up RL algorithms on Atari games by reading manuals released by the Atari game developers. Our framework consists of a QA Extraction module that extracts and summarizes relevant information from the manual and a Reasoning module that evaluates object-agent interactions based on information from the manual. An auxiliary reward is then provided to a standard A2C RL agent, when interaction is detected. Experimentally, various RL algorithms obtain significant improvement in performance and training speed when assisted by our design.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Builds on 54 of its 72 references that are in the map. First posted on arXiv 9 Feb 2023.

Builds on

54 of 72 references.

References · 54

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2017Proximal Policy Optimization Algorithmscited by 14,441
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2017Mask R-CNNcited by 7,259
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  8. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  9. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  10. 2020Longformer: The Long-Document Transformercited by 3,362
  11. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  12. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
38 more
  1. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  3. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  4. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  5. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  6. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  7. 2016Simple Online and Realtime Trackingcited by 862
  8. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  9. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  10. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  11. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  12. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  13. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  14. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  15. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
  16. 2018BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learningcited by 238
  17. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 238
  18. 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190
  19. 2021Episodic Transformer for Vision-and-Language Navigationcited by 155
  20. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  21. 2019Unsupervised State Representation Learning in Ataricited by 144
  22. 2021FILM: Following Instructions in Language with Modular Methodscited by 119
  23. 2022Language Models with Image Descriptors are Strong Few-Shot Video-Language Learnerscited by 117
  24. 2019Using Natural Language for Reward Shaping in Reinforcement Learningcited by 114
  25. 2017Grounded Language Learning in a Simulated 3D Worldcited by 100
  26. 2017Mapping Instructions and Visual Observations to Actions with Reinforcement Learningcited by 80
  27. 2018Learning to Understand Goal Specifications by Modelling Rewardcited by 80
  28. 2017Gated-Attention Architectures for Task-Oriented Language Groundingcited by 72
  29. 2020Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learningcited by 60
  30. 2015Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequencescited by 50
  31. 2021Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learningcited by 45
  32. 2016Learning Language Games through Interactioncited by 43
  33. 2021General-Purpose Question-Answering with Macawcited by 40
  34. 2014Learning to Win by Reading Manuals in a Monte-Carlo Frameworkcited by 39
  35. 2019rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorchcited by 38
  36. 2019RTFM: Generalising to Novel Environment Dynamics via Readingcited by 33
  37. 2017Representation Learning for Grounded Spatial Reasoningcited by 23
  38. 2021Learning Interactive Driving Policies via Data-driven Simulationcited by 17

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 23; 2025: 8; 2026: 5.

45 papers.

Sort:

By area:

  1. 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learningcited by 153
  2. 2023SmartPlay: A Benchmark for LLMs as Intelligent Agentscited by 123
  3. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 114
  4. 2024LLaRA: Supercharging Robot Learning Data for Vision-Language Policycited by 51
  5. 2023Large Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergencecited by 42
  6. 2023Open-Ended Instructable Embodied Agents with Memory-Augmented Large Language Modelscited by 37
  7. 2023OCAtari: Object-Centric Atari 2600 Reinforcement Learning Environmentscited by 30
  8. 2024Interpretable Concept Bottlenecks to Align Reinforcement Learning Agentscited by 23
  9. 2023Semantic HELM: A Human-Readable Memory for Reinforcement Learningcited by 17
  10. 2024Language Grounded Multi-agent Reinforcement Learning with Human-interpretable Communicationcited by 17
  11. 2024Language-Guided World Models: A Model-Based Approach to AI Controlcited by 12
  12. 2024AgentKit: Structured LLM Reasoning with Dynamic Graphscited by 12
33 more
  1. 2023SPRING: Studying the Paper and Reasoning to Play Gamescited by 11
  2. 2024How Well Can Vision Language Models See Image Details?cited by 10
  3. 2024Online Intrinsic Rewards for Decision Making Agents from Large Language Model Feedbackcited by 10
  4. 2023Towards Generalist Robots: A Promising Paradigm via Generative Simulationcited by 9
  5. 2024Enhancing Agent Learning through World Dynamics Modelingcited by 9
  6. 2023Intrinsic Language-Guided Exploration for Complex Long-Horizon Robotic Manipulation Taskscited by 8
  7. 2024Predicting and Understanding Human Action Decisions: Insights from Large Language Models and Cognitive Instance-Based Learningcited by 8
  8. 2024Atari-GPT: Benchmarking Multimodal Large Language Models as Low-Level Policies in Atari Gamescited by 8
  9. 2024Assessing the Zero-Shot Capabilities of LLMs for Action Evaluation in RLcited by 7
  10. 2024Zero-shot Model-based Reinforcement Learning using Large Language Modelscited by 6
  11. 2024Understanding What Affects the Generalization Gap in Visual Reinforcement Learning: Theory and Empirical Evidencecited by 5
  12. 2024Towards Socially and Morally Aware RL agent: Reward Design With LLMcited by 4
  13. 2024Read to Play (R2-Play): Decision Transformer with Multimodal Game Instructioncited by 4
  14. 2024Interpretable end-to-end Neurosymbolic Reinforcement Learning agentscited by 4
  15. 2025LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Modelscited by 4
  16. 2026SCALAR: Learning and Composing Skills through LLM Guided Symbolic Planning and Deep RL Groundingcited by 4
  17. 2024From Reward Shaping to Q-Shaping: Achieving Unbiased Learning with LLM-Guided Knowledgecited by 3
  18. 2024Enhancing Q-Learning with Large Language Model Heuristicscited by 2
  19. 2024Data to Defense: The Role of Curation in Customizing LLMs Against Jailbreaking Attackscited by 2
  20. 2025MoColl: Agent-Based Specific and General Model Collaboration for Image Captioningcited by 2
  21. 2025AED: Automatic Discovery of Effective and Diverse Vulnerabilities for Autonomous Driving Policy with Large Language Modelscited by 2
  22. 2025LLM-Guided Reinforcement Learning: Addressing Training Bottlenecks through Policy Modulationcited by 2
  23. 2024Enhancing Two-Player Performance Through Single-Player Knowledge Transfer: An Empirical Study on Atari 2600 Gamescited by 1
  24. 2025Towards Automated Semantic Interpretability in Reinforcement Learning via Vision-Language Modelscited by 1
  25. 2025Agentic Episodic Controlcited by 1
  26. 2026Balancing Multiple Objectives in Urban Traffic Control with Reinforcement Learning from AI Feedbackcited by 1
  27. 2026Feynman: Knowledge-Infused Diagramming Agent for Scalable Visual Designscited by 1
  28. 2024Towards a Research Community in Interpretable Reinforcement Learning: the InterpPol Workshopcited by 0
  29. 2024OCALM: Object-Centric Assessment with Language Modelscited by 0
  30. 2025APF+: Boosting adaptive-potential function reinforcement learning methods with a W-shaped network for high-dimensional gamescited by 0
  31. 2025Multiple Weaks Win Single Strong: Large Language Models Ensemble Weak Reinforcement Learning Agents into a Supreme Onecited by 0
  32. 2026MIRA: Memory-Integrated Reinforcement Learning Agent with Limited LLM Guidancecited by 0
  33. 2026Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skillscited by 0