mlsanityData to 14 Oct 2026

From Laws to Motivation: Guiding Exploration through Law-Based Reasoning and Rewards

arXiv 2411.15891 ↗First posted 24 Nov 2024cs.LGcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Large Language Models (LLMs) and Reinforcement Learning (RL) are two powerful approaches for building autonomous agents. However, due to limited understanding of the game environment, agents often resort to inefficient exploration and trial-and-error, struggling to develop long-term strategies or make decisions. We propose a method that extracts experience from interaction records to model the underlying laws of the game environment, using these experience as internal motivation to guide agents. These experience, expressed in language, are highly flexible and can either assist agents in reasoning directly or be transformed into rewards for guiding training. Our evaluation results in Crafter demonstrate that both RL and LLM agents benefit from these experience, leading to improved overall performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 14 of its 30 references that are in the map. First posted on arXiv 24 Nov 2024.

Builds on

14 of 30 references.

Closest ancestors · 1

Same direction, newest first

  1. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 13

  1. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  2. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  3. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  4. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
  5. 2017Deep Reinforcement Learning: An Overviewcited by 333
  6. 2019Challenges of Real-World Reinforcement Learningcited by 291
  7. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 289
  8. 2023Reward Design with Language Modelscited by 261
  9. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 200
  10. 2022A Survey of Vision-Language Pre-Trained Modelscited by 177
  11. 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learningcited by 160
  12. 2023OMNI: Open-endedness via Models of human Notions of Interestingnesscited by 70
1 more
  1. 2023Adapting LLM Agents with Universal Feedback in Communicationcited by 19

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Guiding Exploration in Reinforcement Learning Through LLM-Augmented Observations
  2. 2026Beyond Reward Engineering: A Data Recipe for Long-Context Reinforcement Learning
  3. 2026Rethinking Agentic Reinforcement Learning In Large Language Models
  4. 2023Motif: Intrinsic Motivation from Artificial Intelligence Feedback
  5. 2024Words as Beacons: Guiding RL Agents with High-Level Language Prompts
  6. 2025Think in Games: Learning to Reason in Games via Reinforcement Learning with Large Language Models
  7. 2025Agent-RLVR: Training Software Engineering Agents via Guidance and Environment Rewards
  8. 2026Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments?