mlsanityData to 7 Oct 2026

Natural Language Reinforcement Learning

arXiv 2402.07157 ↗First posted 11 Feb 2024cs.CL cs.AI cs.LGcited by 17

Not on the map. Why?

Abstract (arXiv)

Reinforcement Learning (RL) has shown remarkable abilities in learning policies for decision-making tasks. However, RL is often hindered by issues such as low sample efficiency, lack of interpretability, and sparse supervision signals. To tackle these limitations, we take inspiration from the human learning process and introduce Natural Language Reinforcement Learning (NLRL), which innovatively combines RL principles with natural language representation. Specifically, NLRL redefines RL concepts like task objectives, policy, value function, Bellman equation, and policy iteration in natural language space. We present how NLRL can be practically implemented with the latest advancements in large language models (LLMs) like GPT-4. Initial experiments over tabular MDPs demonstrate the effectiveness, efficiency, and also interpretability of the NLRL framework.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 50 of its 77 references that are in the map. First posted on arXiv 11 Feb 2024.

Builds on

50 of 77 references.

References · 50

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2017Proximal Policy Optimization Algorithmscited by 14,441
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  8. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  9. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  10. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  11. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  12. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
38 more
  1. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  2. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  3. 2022Emergent Abilities of Large Language Modelscited by 2,483
  4. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  5. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  6. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  7. 2023PaLM 2 Technical Reportcited by 1,154
  8. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  9. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  10. 2023Large Language Models as Optimizerscited by 830
  11. 2023Benchmarking Large Language Models for News Summarizationcited by 444
  12. 2016Successor Features for Transfer in Reinforcement Learningcited by 331
  13. 2023PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimizationcited by 311
  14. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Trainingcited by 291
  15. 2023Reward Design with Language Modelscited by 257
  16. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 256
  17. 2023Secrets of RLHF in Large Language Models Part I: PPOcited by 226
  18. 2023Large Language Models for Generative Information Extraction: A Surveycited by 183
  19. 2019OpenSpiel: A Framework for Reinforcement Learning in Gamescited by 180
  20. 2023Generative Judge for Evaluating Alignmentcited by 163
  21. 2020What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Studycited by 157
  22. 2023Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learningcited by 155
  23. 2023RoboCat: A Self-Improving Generalist Agent for Robotic Manipulationcited by 108
  24. 2016Model-Free Episodic Controlcited by 100
  25. 2018Batch Active Preference-Based Learning of Reward Functionscited by 87
  26. 2023LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Modelscited by 84
  27. 2023ChessGPT: Bridging Policy Learning and Language Modelingcited by 73
  28. 2024LLM-based NLG Evaluation: Current Status and Challengescited by 71
  29. 2023TIGERScore: Towards Building Explainable Metric for All Text Generation Taskscited by 69
  30. 2022Retrieval-Augmented Reinforcement Learningcited by 53
  31. 2020Learning Rewards from Linguistic Feedbackcited by 41
  32. 2023Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZerocited by 41
  33. 2020Bridging the Gap: Providing Post-Hoc Symbolic Explanations for Sequential Decision-Making Problems with Inscrutable Representationscited by 32
  34. 2023LLF-Bench: Benchmark for Interactive Learning from Language Feedbackcited by 30
  35. 2023Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoningcited by 30
  36. 2023State2Explanation: Concept-Based Explanations to Benefit Agent Learning and User Understandingcited by 17
  37. 2022Spatial-temporal Concept based Explanation of 3D ConvNetscited by 12
  38. 2023Can language agents be alternatives to PPO? A Preliminary Empirical Study On OpenAI Gymcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 8; 2026: 7.

17 papers.

Sort:

By area:

  1. 2025Memory in the Age of AI Agentscited by 285
  2. 2026Reinforcement Learning via Self-Distillationcited by 285
  3. 2025SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learningcited by 89
  4. 2026Expanding the Capabilities of Reinforcement Learning via Text Feedbackcited by 49
  5. 2025Feedback Descent: Open-Ended Text Optimization via Pairwise Comparisoncited by 42
  6. 2025Teaching Language Models to Critique via Reinforcement Learningcited by 38
  7. 2024Learning to Discuss Strategically: A Case Study on One Night Ultimate Werewolfcited by 22
  8. 2025LLM Economist: Large Population Models and Mechanism Design in Multi-Agent Generative Simulacracited by 20
  9. 2025debug-gym: A Text-Based Environment for Interactive Debuggingcited by 19
  10. 2024A Survey On Enhancing Reinforcement Learning in Complex Environments: Insights from Human and LLM Feedbackcited by 9
  11. 2025CMCTS: A Constrained Monte Carlo Tree Search Framework for Mathematical Reasoning in Large Language Modelcited by 9
  12. 2026Learning from Language Feedback via Variational Policy Distillationcited by 6
5 more
  1. 2025Sophia: A Persistent Agent Framework of Artificial Lifecited by 3
  2. 2026ALIVE: Awakening LLM Reasoning via Adversarial Learning and Instructive Verbal Evaluationcited by 1
  3. 2026PaperGuide: Making Small Language-Model Paper-Reading Agents More Efficientcited by 0
  4. 2026Language-Critique Imitation Learning from Suboptimal Demonstrationscited by 0
  5. 2026Learning Stateful Predictive Knowledge From Experiencecited by 0