mlsanityData to 14 Oct 2026

RAMP: Hybrid DRL for Online Learning of Numeric Action Models

arXiv 2604.08685 ↗First posted 9 Apr 2026cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Automated planning algorithms require an action model specifying the preconditions and effects of each action, but obtaining such a model is often hard. Learning action models from observations is feasible, but existing algorithms for numeric domains are offline, requiring expert traces as input. We propose the Reinforcement learning, Action Model learning, and Planning (RAMP) strategy for learning numeric planning action models online via interactions with the environment. RAMP simultaneously trains a Deep Reinforcement Learning (DRL) policy, learns a numeric action model from past interactions, and uses that model to plan future actions when possible. These components form a positive feedback loop: the RL policy gathers data to refine the action model, while the planner generates plans to continue training the RL policy. To facilitate this integration of RL and numeric planning, we developed Numeric PDDLGym, an automated framework for converting numeric planning problems to Gym environments. Experimental results on standard IPC numeric domains show that RAMP significantly outperforms PPO, a well-known DRL algorithm, in terms of solvability and plan quality.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 13 of its 46 references that are in the map. First posted on arXiv 9 Apr 2026.

Builds on

13 of 46 references.

References · 13

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  3. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  4. 2020A Closer Look at Invalid Action Masking in Policy Gradient Algorithmscited by 171
  5. 2019Mastering Complex Control in MOBA Games with Deep Reinforcement Learningcited by 161
  6. 2017Classical Planning in Deep Latent Space: Bridging the Subsymbolic-Symbolic Boundarycited by 58
  7. 2020PDDLGym: Gym Environments from PDDL Problemscited by 45
  8. 2025LLMs as Planning Formalizers: A Survey for Leveraging Large Language Models to Construct Automated Planning Modelscited by 38
  9. 2017Efficient, Safe, and Probably Approximately Complete Learning of Action Modelscited by 16
  10. 2021Safe Learning of Lifted Action Modelscited by 16
  11. 2023Autonomous Capability Assessment of Sequential Decision-Making Systems in Stochastic Settings (Extended Version)cited by 6
  12. 2023Polycraft World AI Lab (PAL): An Extensible Platform for Evaluating Artificial Intelligence Agentscited by 1
1 more
  1. 2025Toward PDDL Planning Copilotcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Integrating Reinforcement Learning, Action Model Learning, and Numeric Planning for Tackling Complex Tasks
  2. 2024Action Mapping for Reinforcement Learning in Continuous Environments with Constraints
  3. 2024Efficient Exploration in Deep Reinforcement Learning: A Novel Bayesian Actor-Critic Algorithm
  4. 2025Taxonomy and Trends in Reinforcement Learning for Robotics and Control Systems: A Structured Review
  5. 2025LLM-Guided Reinforcement Learning: Addressing Training Bottlenecks through Policy Modulation
  6. 2025Scalable Offline Model-Based RL with Action Chunks
  7. 2026CAST: Alternating State-Value Targets and Expanded Policy Gradients for Model-Based Reinforcement Learning
  8. 2025Context-Sensitive Abstractions for Reinforcement Learning with Parameterized Actions