mlsanityData to 14 Oct 2026

Using Natural Language for Reward Shaping in Reinforcement Learning

arXiv 1903.02020 ↗First posted 5 Mar 2019cs.LG cs.AI stat.MLcited by 114

Not on the map. Why?

Cited by 22 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent reinforcement learning (RL) approaches have shown strong performance in complex domains such as Atari games, but are often highly sample inefficient. A common approach to reduce interaction time with the environment is to use reward shaping, which involves carefully designing reward functions that provide the agent intermediate rewards for progress towards the goal. However, designing appropriate shaping rewards is known to be difficult as well as time-consuming. In this work, we address this problem by using natural language instructions to perform reward shaping. We propose the LanguagE-Action Reward Network (LEARN), a framework that maps free-form natural language instructions to intermediate rewards based on actions taken by the agent. These intermediate language-based rewards can seamlessly be integrated into any standard reinforcement learning algorithm. We experiment with Montezuma's Revenge from the Atari Learning Environment, a popular benchmark in RL. Our experiments on a diverse set of 15 tasks demonstrate that, for the same number of interactions with the environment, language-based rewards lead to successful completion of the task 60% more often on average, compared to learning without language.

Key facts. Cited by 114 papers in the map (data through 14 Oct 2026). First posted on arXiv 5 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 3; 2021: 13; 2022: 17; 2023: 26; 2024: 29; 2025: 17; 2026: 9.

114 papers.

Sort:

By area:

  1. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
  2. 2026Reinforcement Learning via Self-Distillationcited by 332
  3. 2023Language to Rewards for Robotic Skill Synthesiscited by 312
  4. 2023Reward Design with Language Modelscited by 261
  5. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 209
  6. 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Drivingcited by 190
  7. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  8. 2023Training Language Models with Language Feedback at Scalecited by 128
  9. 2021Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationcited by 118
  10. 2024VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thoughtcited by 55
  11. 2022Semantic Exploration from Language Abstractions and Pretrained Representationscited by 51
  12. 2022Improving Intrinsic Exploration with Language Abstractionscited by 49
38 more
  1. 2022Training Language Models with Language Feedbackcited by 49
  2. 2023Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manualscited by 45
  3. 2021ELLA: Exploration through Learned Language Abstractioncited by 42
  4. 2020Learning Rewards from Linguistic Feedbackcited by 41
  5. 2024Comprehensive Overview of Reward Engineering and Shaping in Advancing Reinforcement Learning Applicationscited by 38
  6. 2023AlignDiff: Aligning Diverse Human Preferences via Behavior-Customisable Diffusion Modelcited by 37
  7. 2021Tell me why! Explanations support learning relational and causal structurecited by 36
  8. 2022Fixing Model Bugs with Natural Language Patchescited by 33
  9. 2021Direct Behavior Specification via Constrained Reinforcement Learningcited by 31
  10. 2023Interactive Natural Language Processingcited by 31
  11. 2026Aligning Language Models from User Interactionscited by 28
  12. 2021Interactive Learning from Activity Descriptioncited by 26
  13. 2022Leveraging Language for Accelerated Learning of Tool Manipulationcited by 26
  14. 2022EAGER: Asking and Answering Questions for Automatic Reward Shaping in Language-guided RLcited by 23
  15. 2022Law Informs Code: A Legal Informatics Approach to Aligning Artificial Intelligence with Humanscited by 23
  16. 2022Leveraging Approximate Symbolic Models for Reinforcement Learning via Skill Diversitycited by 22
  17. 2022How to talk so AI will learn: Instructions, descriptions, and autonomycited by 22
  18. 2023Accelerating Reinforcement Learning of Robotic Manipulations via Feedback from Large Language Modelscited by 22
  19. 2024Beyond Sparse Rewards: Enhancing Reinforcement Learning with Language Model Critique in Text Generationcited by 22
  20. 2024CurricuLLM: Automatic Task Curricula Design for Learning Complex Robot Skills using Large Language Modelscited by 22
  21. 2024Natural Language Can Help Bridge the Sim2Real Gapcited by 21
  22. 2024DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RLcited by 21
  23. 2020Inverse Reinforcement Learning with Natural Language Goalscited by 20
  24. 2025Leveraging Large Language Models for Effective and Explainable Multi-Agent Credit Assignmentcited by 18
  25. 2022Using Both Demonstrations and Language Instructions to Efficiently Learn Robotic Taskscited by 17
  26. 2023Semantic HELM: A Human-Readable Memory for Reinforcement Learningcited by 17
  27. 2023State2Explanation: Concept-Based Explanations to Benefit Agent Learning and User Understandingcited by 17
  28. 2024Language Grounded Multi-agent Reinforcement Learning with Human-interpretable Communicationcited by 17
  29. 2021Zero-shot Task Adaptation using Natural Languagecited by 15
  30. 2024The RL/LLM Taxonomy Tree: Reviewing Synergies Between Reinforcement Learning and Large Language Modelscited by 15
  31. 2023f-Policy Gradients: A General Framework for Goal Conditioned RL using f-Divergencescited by 14
  32. 2021Choice Set Misspecification in Reward Inferencecited by 13
  33. 2023Active Reward Learning from Multiple Teacherscited by 13
  34. 2024Potential-Based Reward Shaping For Intrinsic Motivationcited by 12
  35. 2021Embedding Symbolic Temporal Knowledge into Deep Sequential Modelscited by 11
  36. 2023SPRING: Studying the Paper and Reasoning to Play Gamescited by 11
  37. 2024Learning with Language-Guided State Abstractionscited by 11
  38. 2024Reward Machines for Deep RL in Noisy and Uncertain Environmentscited by 11

50 most cited of 114 All 114 (JSON)