mlsanityData to 7 Oct 2026

SLOPE: Optimistic Potential Landscape Shaping for Model-based Reinforcement Learning

arXiv 2602.03201 ↗First posted 3 Feb 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Model-based reinforcement learning (MBRL) is sample-efficient but struggles in sparse reward settings. A critical bottleneck arises from the lack of informative gradients in sparse settings, where standard reward models often yield flat landscapes that struggle to guide planning. To address this challenge, we propose Shaping Landscapes with Optimistic Potential Estimates (SLOPE), a novel framework that shifts reward modeling from predicting sparse scalars to constructing informative potential landscapes. SLOPE employs optimistic distributional regression to estimate high-confidence upper bounds, which amplifies rare success signals and ensures sufficient exploration gradients. Evaluations on 30+ tasks across 5 benchmarks and real-world robotic deployments, demonstrate that SLOPE consistently outperforms leading baselines in fully sparse, semi-sparse, and dense rewards.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 32 references that are in the map. First posted on arXiv 3 Feb 2026.

Builds on

27 of 32 references.

References · 27

  1. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  2. 2017Hindsight Experience Replaycited by 1,096
  3. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  4. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  5. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  6. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
  7. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  8. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 216
  9. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  10. 2022Masked World Models for Visual Controlcited by 169
  11. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learningcited by 153
  12. 2015Model Predictive Path Integral Control using Covariance Variable Importance Samplingcited by 122
15 more
  1. 2018Episodic Curiosity through Reachabilitycited by 109
  2. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 107
  3. 2022MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrationscited by 59
  4. 2022Improving Intrinsic Exploration with Language Abstractionscited by 49
  5. 2022Graph Inverse Reinforcement Learning from Diverse Videoscited by 45
  6. 2023MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulationcited by 27
  7. 2023A Study of Global and Episodic Bonuses for Exploration in Contextual MDPscited by 19
  8. 2023Reinforcement Learning with Foundation Priors: Let the Embodied Agent Efficiently Learn on Its Owncited by 16
  9. 2025Episodic Novelty Through Temporal Distancecited by 15
  10. 2024Open-World Reinforcement Learning over Long Short-Term Imaginationcited by 13
  11. 2025Bootstrapped Model Predictive Controlcited by 13
  12. 2022Exploring through Random Curiosity with General Value Functionscited by 10
  13. 2025Multi-Stage Manipulation with Demonstration-Augmented Reward, Policy, and World Model Learningcited by 10
  14. 2023DreamSmooth: Improving Model-based Reinforcement Learning via Reward Smoothingcited by 8
  15. 2025Bootstrapped Reward Shapingcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Optimistic World Models: Efficient Exploration in Model-Based Deep Reinforcement Learning
  2. 2020Learning to Plan Optimistically: Uncertainty-Guided Deep Exploration via Latent Model Ensembles
  3. 2025General Exploratory Bonus for Optimistic Exploration in RLHF
  4. 2026Escaping Offline Pessimism: Vector-Field Reward Shaping for Safe Frontier Exploration
  5. 2024Model-based RL as a Minimalist Approach to Horizon-Free and Second-Order Bounds
  6. 2020Shaping Rewards for Reinforcement Learning with Imperfect Demonstrations using Generative Models
  7. 2018Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees
  8. 2022On Practical Reinforcement Learning: Provable Robustness, Scalability, and Statistical Efficiency