mlsanityData to 7 Oct 2026

A Model-Based Approach for Improving Reinforcement Learning Efficiency Leveraging Expert Observations

arXiv 2402.18836 ↗First posted 29 Feb 2024cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper investigates how to incorporate expert observations (without explicit information on expert actions) into a deep reinforcement learning setting to improve sample efficiency. First, we formulate an augmented policy loss combining a maximum entropy reinforcement learning objective with a behavioral cloning loss that leverages a forward dynamics model. Then, we propose an algorithm that automatically adjusts the weights of each component in the augmented loss function. Experiments on a variety of continuous control tasks demonstrate that the proposed algorithm outperforms various benchmarks by effectively utilizing available expert observations.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 43 references that are in the map. First posted on arXiv 29 Feb 2024.

Builds on

36 of 43 references.

References · 36

  1. 2015Continuous control with deep reinforcement learningcited by 3,279
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2016Generative Adversarial Imitation Learningcited by 1,576
  4. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  5. 2018DeepMind Control Suitecited by 935
  6. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  7. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  8. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  9. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  10. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  11. 2018Behavioral Cloning from Observationcited by 482
  12. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
24 more
  1. 2018Maximum a Posteriori Policy Optimisationcited by 318
  2. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 256
  3. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 228
  4. 2019Deep Dynamics Models for Learning Dexterous Manipulationcited by 200
  5. 2018Generative Adversarial Imitation from Observationcited by 183
  6. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  7. 2016Learning to Poke by Poking: Experiential Learning of Intuitive Physicscited by 152
  8. 2018Imitating Latent Policies from Observationcited by 101
  9. 2019Provably Efficient Imitation Learning from Observation Alonecited by 78
  10. 2018Reinforcement Learning from Imperfect Demonstrationscited by 74
  11. 2020A Game Theoretic Framework for Model Based Reinforcement Learningcited by 72
  12. 2021Imitation Learning: Progress, Taxonomies and Challengescited by 56
  13. 2021MobILE: Model-Based Imitation Learning From Observation Alonecited by 44
  14. 2021Generalized Proximal Policy Optimization with Sample Reusecited by 39
  15. 2016Model-based Adversarial Imitation Learningcited by 29
  16. 2021Offline Reinforcement Learning with Soft Behavior Regularizationcited by 28
  17. 2018Fast Policy Learning through Imitation and Reinforcementcited by 26
  18. 2018Inverse reinforcement learning for video gamescited by 26
  19. 2023Visual Imitation Learning with Patch Rewardscited by 16
  20. 2023Adversarial Imitation Learning from Visual Observations using Latent Informationcited by 10
  21. 2023Optimal Transport Perturbations for Safe Reinforcement Learning with Robustness Guaranteescited by 9
  22. 2022Combining imitation and deep reinforcement learning to accomplish human-level performance on a virtual foraging taskcited by 4
  23. 2022Opportunities and Challenges from Using Animal Videos in Reinforcement Learning for Navigationcited by 4
  24. 2017Learning Policies for Markov Decision Processes from Datacited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Data augmentation for efficient learning from parametric experts
  2. 2022A Few Expert Queries Suffices for Sample-Efficient RL with Resets and Linear Value Approximation
  3. 2021A Closer Look at Advantage-Filtered Behavioral Cloning in High-Noise Datasets
  4. 2024Inverse Delayed Reinforcement Learning
  5. 2022Efficient Reinforcement Learning from Demonstration Using Local Ensemble and Reparameterization with Split and Merge of Expert Policies
  6. 2023Demonstration-Regularized RL
  7. 2025Statistical analysis of Inverse Entropy-regularized Reinforcement Learning
  8. 2020Contextual Policy Transfer in Reinforcement Learning Domains via Deep Mixtures-of-Experts