mlsanityData to 14 Oct 2026

Which Experiences Are Influential for Your Agent? Policy Iteration with Turn-over Dropout

arXiv 2301.11168 ↗First posted 26 Jan 2023cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

In reinforcement learning (RL) with experience replay, experiences stored in a replay buffer influence the RL agent's performance. Information about the influence is valuable for various purposes, including experience cleansing and analysis. One method for estimating the influence of individual experiences is agent comparison, but it is prohibitively expensive when there is a large number of experiences. In this paper, we present PI+ToD as a method for efficiently estimating the influence of experiences. PI+ToD is a policy iteration that efficiently estimates the influence of experiences by utilizing turn-over dropout. We demonstrate the efficiency of PI+ToD with experiments in MuJoCo environments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 41 of its 54 references that are in the map. First posted on arXiv 26 Jan 2023.

Builds on

41 of 54 references.

References · 41

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2016Layer Normalizationcited by 5,669
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  5. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  6. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  7. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  8. 2016OpenAI Gymcited by 1,736
  9. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,552
  10. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  11. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  12. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
29 more
  1. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  2. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 288
  3. 2022The Primacy Bias in Deep Reinforcement Learningcited by 247
  4. 2018Representer Point Selection for Explaining Deep Neural Networkscited by 187
  5. 2020Revisiting Fundamentals of Experience Replaycited by 175
  6. 2020Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Criticscited by 165
  7. 2019On the Accuracy of Influence Functions for Measuring Group Effectscited by 147
  8. 2017Visualizing and Understanding Atari Agentscited by 141
  9. 2021Scaling Up Influence Functionscited by 132
  10. 2020FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debuggingcited by 120
  11. 2021Dropout Q-Functions for Doubly Efficient Reinforcement Learningcited by 114
  12. 2017A Deeper Look at Experience Replaycited by 105
  13. 2019Towards Interpretable Reinforcement Learning Using Attention Augmented Agentscited by 82
  14. 2018SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planningcited by 78
  15. 2020Transient Non-Stationarity and Generalisation in Deep Reinforcement Learningcited by 75
  16. 2020DisCor: Corrective Feedback in Reinforcement Learning via Distribution Correctioncited by 71
  17. 2016Graying the black box: Understanding DQNscited by 68
  18. 2019Data Cleansing for Models Trained with SGDcited by 67
  19. 2021DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularizationcited by 66
  20. 2018Toward Interpretable Deep Reinforcement Learning with Linear Model U-Treescited by 65
  21. 2018PEORL: Integrating Symbolic Planning and Hierarchical Reinforcement Learning for Robust Decision-Makingcited by 52
  22. 2016Towards Deep Symbolic Reinforcement Learningcited by 42
  23. 2021HYDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networkscited by 38
  24. 2021Regret Minimization Experience Replay in Off-Policy Reinforcement Learningcited by 25
  25. 2017The Effects of Memory Replay in Reinforcement Learningcited by 23
  26. 2020Efficient Estimation of Influence of a Training Instancecited by 11
  27. 2022Analysis of Stochastic Processes through Replay Bufferscited by 9
  28. 2017Active Exploration for Learning Symbolic Representationscited by 8
  29. 2021Sim and Real: Better Togethercited by 7

Built on by

Not cited yet.