mlsanityData to 7 Oct 2026

External Model Motivated Agents: Reinforcement Learning for Enhanced Environment Sampling

arXiv 2407.00264 ↗First posted 28 Jun 2024cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Unlike reinforcement learning (RL) agents, humans remain capable multitaskers in changing environments. In spite of only experiencing the world through their own observations and interactions, people know how to balance focusing on tasks with learning about how changes may affect their understanding of the world. This is possible by choosing to solve tasks in ways that are interesting and generally informative beyond just the current task. Motivated by this, we propose an agent influence framework for RL agents to improve the adaptation efficiency of external models in changing environments without any changes to the agent's rewards. Our formulation is composed of two self-contained modules: interest fields and behavior shaping via interest fields. We implement an uncertainty-based interest field algorithm as well as a skill-sampling-based behavior-shaping algorithm to use in testing this framework. Our results show that our method outperforms the baselines in terms of external model adaptation on metrics that measure both efficiency and performance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 44 references that are in the map. First posted on arXiv 28 Jun 2024.

Builds on

25 of 44 references.

References · 25

  1. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2018Exploration by Random Network Distillationcited by 966
  4. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 813
  5. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  6. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  7. 2020A Survey of Deep Active Learningcited by 547
  8. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
  9. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
  10. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 297
  11. 2019Self-Supervised Exploration via Disagreementcited by 263
  12. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
13 more
  1. 2021A Practical Guide to Multi-Objective Reinforcement Learning and Planningcited by 240
  2. 2014A Survey of Multi-Objective Sequential Decision-Makingcited by 229
  3. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
  4. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
  5. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 107
  6. 2022Deep Hierarchical Planning from Pixelscited by 96
  7. 2019A survey on intrinsic motivation in reinforcement learningcited by 79
  8. 2020Reinforcement Learning for Non-Stationary Markov Decision Processes: The Blessing of (More) Optimismcited by 75
  9. 2020Intrinsic Reward Driven Imitation Learning via Generative Modelcited by 35
  10. 2021Sample Efficient Reinforcement Learning via Model-Ensemble Exploration and Exploitationcited by 17
  11. 2020A Provably Efficient Sample Collection Strategy for Reinforcement Learningcited by 15
  12. 2022NovGrid: A Flexible Grid World for Evaluating Agent Response to Noveltycited by 14
  13. 2022Rewarding Episodic Visitation Discrepancy for Exploration in Reinforcement Learningcited by 7

Built on by

Not cited yet.