mlsanityData to 7 Oct 2026

Combining imitation and deep reinforcement learning to accomplish human-level performance on a virtual foraging task

arXiv 2203.06250 ↗First posted 11 Mar 2022cs.LG cs.SY eess.SYcited by 4

Not on the map. Why?

Abstract (arXiv)

We develop a simple framework to learn bio-inspired foraging policies using human data. We conduct an experiment where humans are virtually immersed in an open field foraging environment and are trained to collect the highest amount of rewards. A Markov Decision Process (MDP) framework is introduced to model the human decision dynamics. Then, Imitation Learning (IL) based on maximum likelihood estimation is used to train Neural Networks (NN) that map human decisions to observed states. The results show that passive imitation substantially underperforms humans. We further refine the human-inspired policies via Reinforcement Learning (RL) using the on-policy Proximal Policy Optimization (PPO) algorithm which shows better stability than other algorithms and can steadily improve the policies pretrained with IL. We show that the combination of IL and RL can match human results and that good performance strongly depends on combining the allocentric information with an egocentric representation of the environment.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 24 of its 64 references that are in the map. First posted on arXiv 11 Mar 2022.

Builds on

24 of 64 references.

References · 24

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2022Generative Adversarial Networkscited by 5,603
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  5. 2015Trust Region Policy Optimizationcited by 2,503
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  7. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  8. 2016Generative Adversarial Imitation Learningcited by 1,576
  9. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  10. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  11. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  12. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
12 more
  1. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
  2. 2019Challenges of Real-World Reinforcement Learningcited by 282
  3. 2019A Divergence Minimization Perspective on Imitation Learning Methodscited by 188
  4. 2020Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPOcited by 184
  5. 2020What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Studycited by 157
  6. 2014Reinforcement and Imitation Learning via Interactive No-Regret Learningcited by 116
  7. 2018Truncated Horizon Policy Search: Combining Reinforcement Learning & Imitation Learningcited by 39
  8. 2021Generalized Proximal Policy Optimization with Sample Reusecited by 39
  9. 2018Fast Policy Learning through Imitation and Reinforcementcited by 26
  10. 2020Guided Exploration with Proximal Policy Optimization using a Single Demonstrationcited by 19
  11. 2020Uncertainty-Aware Policy Optimization: A Robust, Adaptive Trust Region Approachcited by 6
  12. 2020Forward and inverse reinforcement learning sharing network weights and hyperparameterscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 1; 2026: 1.

4 papers.

  1. 2023Adversarial Imitation Learning from Visual Observations using Latent Informationcited by 10
  2. 2025Reinforcement Learning for Active Mattercited by 4
  3. 2024A Model-Based Approach for Improving Reinforcement Learning Efficiency Leveraging Expert Observationscited by 0
  4. 2026Physics-informed Goal-Conditioned Reinforcement Learning under Hybrid Contact Dynamicscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025An Introduction to Deep Reinforcement and Imitation Learning
  2. 2019Leveraging Human Guidance for Deep Reinforcement Learning Tasks
  3. 2019A New Framework for Query Efficient Active Imitation Learning
  4. 2025Unlocking the Potential of Soft Actor-Critic for Imitation Learning
  5. 2024Reward-free World Models for Online Imitation Learning
  6. 2024Zero-shot Imitation Policy via Search in Demonstration Dataset
  7. 2021Hyperparameter Selection for Imitation Learning
  8. 2023PIRLNav: Pretraining with Imitation and RL Finetuning for ObjectNav