mlsanityData to 7 Oct 2026

Active Exploration for Robotic Manipulation

arXiv 2210.12806 ↗First posted 23 Oct 2022cs.RO cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

Robotic manipulation stands as a largely unsolved problem despite significant advances in robotics and machine learning in recent years. One of the key challenges in manipulation is the exploration of the dynamics of the environment when there is continuous contact between the objects being manipulated. This paper proposes a model-based active exploration approach that enables efficient learning in sparse-reward robotic manipulation tasks. The proposed method estimates an information gain objective using an ensemble of probabilistic models and deploys model predictive control (MPC) to plan actions online that maximize the expected reward while also performing directed exploration. We evaluate our proposed algorithm in simulation and on a real robot, trained from scratch with our method, on a challenging ball pushing task on tilted tables, where the target ball position is not known to the agent a-priori. Our real-world robot experiment serves as a fundamental application of active exploration in model-based reinforcement learning of complex robotic manipulation tasks.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 25 of its 49 references that are in the map. First posted on arXiv 23 Oct 2022.

Builds on

25 of 49 references.

References · 25

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2017Billion-scale similarity search with GPUscited by 2,477
  4. 2016OpenAI Gymcited by 1,684
  5. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  6. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  7. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  8. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  9. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  10. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  11. 2019On Variational Bounds of Mutual Informationcited by 550
  12. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
13 more
  1. 2020Planning to Explore via Self-Supervised World Modelscited by 365
  2. 2020Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Userscited by 336
  3. 2019Self-Supervised Exploration via Disagreementcited by 263
  4. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 207
  5. 2019A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithmscited by 136
  6. 2019Variational Bayesian Optimal Experimental Designcited by 116
  7. 2018Model-Based Active Explorationcited by 105
  8. 2018Temporal Difference Models: Model-Free Deep RL for Model-Based Controlcited by 89
  9. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
  10. 2020Reinforcement Learning through Active Inferencecited by 47
  11. 2017Combining Model-Based and Model-Free Updates for Trajectory-Centric Reinforcement Learningcited by 27
  12. 2021Model Predictive Actor-Critic: Accelerating Robot Skill Acquisition with Deep Reinforcement Learningcited by 25
  13. 2019Curious iLQR: Resolving Uncertainty in Model-based RLcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 1; 2026: 2.

9 papers.

  1. 2024RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulationcited by 48
  2. 2023Diminishing Return of Value Expansion Methods in Model-Based Reinforcement Learningcited by 5
  3. 2026Towards Exploratory and Focused Manipulation with Bimanual Active Perception: A New Problem, Benchmark and Strategycited by 4
  4. 2025Deep Active Inference Agents for Delayed and Long-Horizon Environmentscited by 3
  5. 2024Learning Off-policy with Model-based Intrinsic Motivation For Active Online Explorationcited by 1
  6. 2024Active Inference Meeting Energy-Efficient Control of Parallel and Identical Machinescited by 1
  7. 2023Tactile Active Inference Reinforcement Learning for Efficient Robotic Manipulation Skill Acquisitioncited by 0
  8. 2024Diminishing Return of Value Expansion Methodscited by 0
  9. 2026Sim-to-Real Transfer for Muscle-Actuated Robots via Generalized Actuator Networkscited by 0