mlsanityData to 14 Oct 2026

On-Robot Bayesian Reinforcement Learning for POMDPs

arXiv 2307.11954 ↗First posted 22 Jul 2023cs.RO cs.AI cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Robot learning is often difficult due to the expense of gathering data. The need for large amounts of data can, and should, be tackled with effective algorithms and leveraging expert information on robot dynamics. Bayesian reinforcement learning (BRL), thanks to its sample efficiency and ability to exploit prior knowledge, is uniquely positioned as such a solution method. Unfortunately, the application of BRL has been limited due to the difficulties of representing expert knowledge as well as solving the subsequent inference problem. This paper advances BRL for robotics by proposing a specialized framework for physical systems. In particular, we capture this knowledge in a factored representation, then demonstrate the posterior factorizes in a similar shape, and ultimately formalize the model in a Bayesian framework. We then introduce a sample-based online solution method, based on Monte-Carlo tree search and particle filtering, specialized to solve the resulting model. This approach can, for example, utilize typical low-level robot simulators and handle uncertainty over unknown dynamics of the environment. We empirically demonstrate its efficiency by performing on-robot learning in two human-robot interaction tasks with uncertainty about human behavior, achieving near-optimal performance after only a handful of real-world episodes. A video of learned policies is at https://youtu.be/H9xp60ngOes.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 19 of its 33 references that are in the map. First posted on arXiv 22 Jul 2023.

Builds on

19 of 33 references.

References · 19

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  5. 2020Mastering Atari with Discrete World Modelscited by 1,007
  6. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  7. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  8. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
  9. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  10. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 353
  11. 2017Preparing for the Unknown: Learning a Universal Policy with Online System Identificationcited by 141
  12. 2019End-to-End Robotic Reinforcement Learning without Reward Engineeringcited by 122
7 more
  1. 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 83
  2. 2022Sample Efficient Grasp Learning Using Equivariant Modelscited by 53
  3. 2022On-Robot Learning With Equivariant Modelscited by 42
  4. 2020Discriminative Particle Filter Reinforcement Learning for Complex Partial Observationscited by 28
  5. 2018Learning in POMDPs with Monte Carlo Tree Searchcited by 15
  6. 2018Bayesian Reinforcement Learning in Factored POMDPscited by 15
  7. 2022BADDr: Bayes-Adaptive Deep Dropout RL for POMDPscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2024Belief Aided Navigation using Bayesian Reinforcement Learning for Avoiding Humans in Blind Spotscited by 0
  2. 2025Exploring the Link Between Bayesian Inference and Embodied Intelligence: Toward Open Physical-World Embodied AI Systemscited by 0
  3. 2026Reinforcement Learning from Real-Robot Interaction without High-Quality State Estimatescited by 0