mlsanityData to 14 Oct 2026

DreamSmooth: Improving Model-based Reinforcement Learning via Reward Smoothing

arXiv 2311.01450 ↗First posted 2 Nov 2023cs.LG cs.AI cs.ROcited by 8

Not on the map. Why?

Abstract (arXiv)

Model-based reinforcement learning (MBRL) has gained much attention for its ability to learn complex behaviors in a sample-efficient way: planning actions by generating imaginary trajectories with predicted rewards. Despite its success, we found that surprisingly, reward prediction is often a bottleneck of MBRL, especially for sparse rewards that are challenging (or even ambiguous) to predict. Motivated by the intuition that humans can learn from rough reward estimates, we propose a simple yet effective reward smoothing approach, DreamSmooth, which learns to predict a temporally-smoothed reward, instead of the exact reward at the given timestep. We empirically show that DreamSmooth achieves state-of-the-art performance on long-horizon sparse-reward tasks both in sample efficiency and final performance without losing performance on common benchmarks, such as Deepmind Control Suite and Atari benchmarks.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 21 of its 32 references that are in the map. First posted on arXiv 2 Nov 2023.

Builds on

21 of 32 references.

References · 21

  1. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  2. 2018World Modelscited by 1,513
  3. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  4. 2023Mastering Diverse Domains through World Modelscited by 1,253
  5. 2020Mastering Atari with Discrete World Modelscited by 1,007
  6. 2018DeepMind Control Suitecited by 956
  7. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  8. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  9. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 260
  10. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 200
  11. 2022Masked World Models for Visual Controlcited by 170
  12. 2023Structured World Models from Human Videoscited by 147
9 more
  1. 2020Model-Based Offline Planningcited by 129
  2. 2023Multi-View Masked World Models for Visual Robotic Manipulationcited by 80
  3. 2021DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representationscited by 74
  4. 2020Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstructioncited by 66
  5. 2022Skill-based Model-based Reinforcement Learningcited by 50
  6. 2023Facing Off World Model Backbones: RNNs, Transformers, and S4cited by 31
  7. 2021Structured World Belief for Reinforcement Learning in POMDPcited by 21
  8. 2020Models, Pixels, and Rewards: Evaluating Design Trade-offs in Visual Model-Based Reinforcement Learningcited by 7
  9. 2020A multiscale model of terrain dynamics for real-time earthmoving simulationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 2.

8 papers.

  1. 2025SEEA-R1: Tree-Structured Reinforcement Fine-Tuning for Self-Evolving Embodied Agentscited by 15
  2. 2024Open-World Reinforcement Learning over Long Short-Term Imaginationcited by 13
  3. 2024CompilerDream: Learning a Compiler World Model for General Code Optimizationcited by 6
  4. 2026A Mechanistic View on Video Generation as World Models: State and Dynamicscited by 6
  5. 2025Emoti-Attack: Zero-Perturbation Adversarial Attacks on NLP Systems via Emoji Sequencescited by 5
  6. 2024Representation Learning For Efficient Deep Multi-Agent Reinforcement Learningcited by 3
  7. 2026SLOPE: Optimistic Potential Landscape Shaping for Model-based Reinforcement Learningcited by 1
  8. 2025Learn A Flexible Exploration Model for Parameterized Action Markov Decision Processescited by 0