mlsanityData to 7 Oct 2026

Bayesian Q-learning With Imperfect Expert Demonstrations

arXiv 2210.01800 ↗First posted 1 Oct 2022cs.LG cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Guided exploration with expert demonstrations improves data efficiency for reinforcement learning, but current algorithms often overuse expert information. We propose a novel algorithm to speed up Q-learning with the help of a limited amount of imperfect expert demonstrations. The algorithm avoids excessive reliance on expert data by relaxing the optimal expert assumption and gradually reducing the usage of uninformative expert data. Experimentally, we evaluate our approach on a sparse-reward chain environment and six more complicated Atari games with delayed rewards. With the proposed methods, we can achieve better results than Deep Q-learning from Demonstrations (Hester et al., 2017) in most environments.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 14 of its 29 references that are in the map. First posted on arXiv 1 Oct 2022.

Builds on

14 of 29 references.

References · 14

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  2. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  3. 2015Prioritized Experience Replaycited by 1,070
  4. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  5. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  6. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  7. 2018Is Q-learning Provably Efficient?cited by 485
  8. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  9. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 256
  10. 2018Randomized Prior Functions for Deep Reinforcement Learningcited by 218
  11. 2017Deep Exploration via Randomized Value Functionscited by 155
  12. 2020Temporally-Extended ε-Greedy Explorationcited by 29
2 more
  1. 2019Reinforcement Learning from Imperfect Demonstrations under Soft Expert Guidancecited by 25
  2. 2020Shaping Rewards for Reinforcement Learning with Imperfect Demonstrations using Generative Modelscited by 14

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 0; 2025: 2.

3 papers.

  1. 2023Correcting discount-factor mismatch in on-policy policy gradient methodscited by 9
  2. 2025Uncertainty-Based Smooth Policy Regularisation for Reinforcement Learning with Few Demonstrationscited by 0
  3. 2025Optimizing Algorithms for Mobile Health Interventions with Active Querying Optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Fill in the Blanks: Accelerating Q-Learning with a Handful of Demonstrations in Sparse Reward Settings
  2. 2026Q-based Variational Inverse Reinforcement Learning
  3. 2025Beyond-Expert Performance with Limited Demonstrations: Efficient Imitation Learning with Double Exploration
  4. 2024A Bayesian Solution To The Imitation Gap
  5. 2018Reinforcement Learning from Imperfect Demonstrations
  6. 2024"Give Me an Example Like This": Episodic Active Reinforcement Learning from Demonstrations
  7. 2024Accelerating Inverse Reinforcement Learning with Expert Bootstrapping
  8. 2020Safe Imitation Learning via Fast Bayesian Reward Inference from Preferences