mlsanityData to 14 Oct 2026

Bounded Exploration with World Model Uncertainty in Soft Actor-Critic Reinforcement Learning Algorithm

arXiv 2412.06139 ↗First posted 9 Dec 2024cs.LG cs.SY eess.SYcited by 1

On the map: World models → Latent dynamics models and model-based RL

also in 1 more
Abstract (arXiv)

One of the bottlenecks preventing Deep Reinforcement Learning algorithms (DRL) from real-world applications is how to explore the environment and collect informative transitions efficiently. The present paper describes bounded exploration, a novel exploration method that integrates both 'soft' and intrinsic motivation exploration. Bounded exploration notably improved the Soft Actor-Critic algorithm's performance and its model-based extension's converging speed. It achieved the highest score in 6 out of 8 experiments. Bounded exploration presents an alternative method to introduce intrinsic motivations to exploration when the original reward function has strict meanings.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 8 of its 28 references that are in the map. First posted on arXiv 9 Dec 2024.

Builds on

8 of 28 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Intrinsic Motivation in Model-based Reinforcement Learning: A Brief Review24 Jan
  2. 2018Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees10 Jul
  3. 2018Model-Ensemble Trust-Region Policy Optimization28 Feb

Other references · 5

  1. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  2. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 482
  3. 2018Model-Based Value Estimation for Efficient Model-Free Reinforcement Learningcited by 156
  4. 2019A survey on intrinsic motivation in reinforcement learningcited by 81
  5. 2016Learning and Policy Search in Stochastic Dynamical Systems with Bayesian Neural Networkscited by 49

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Morphology-Conditioned World Model for Cross-Embodiment Quadrupedal Locomotioncited by 4

Similar, not cited

Close in wording, no citation either way

  1. 2025Smart Exploration in Reinforcement Learning using Bounded Uncertainty Models
  2. 2024Efficient Exploration in Deep Reinforcement Learning: A Novel Bayesian Actor-Critic Algorithm
  3. 2024Improving Actor-Critic Training with Steerable Action-Value Approximation Errors
  4. 2025Wasserstein Barycenter Soft Actor-Critic
  5. 2024Risk-Sensitive Soft Actor-Critic for Robust Deep Reinforcement Learning under Distribution Shifts
  6. 2025Learning to Explore in Diverse Reward Settings via Temporal-Difference-Error Maximization
  7. 2025Exploration by Random Reward Perturbation
  8. 2025Exploration from a Primal-Dual Lens: Value-Incentivized Actor-Critic Methods for Sample-Efficient Online RL