mlsanityData to 14 Oct 2026

On the Value of Myopic Behavior in Policy Reuse

arXiv 2305.17623 ↗First posted 28 May 2023cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Leveraging learned strategies in unfamiliar scenarios is fundamental to human intelligence. In reinforcement learning, rationally reusing the policies acquired from other tasks or human experts is critical for tackling problems that are difficult to learn from scratch. In this work, we present a framework called Selective Myopic bEhavior Control~(SMEC), which results from the insight that the short-term behaviors of prior policies are sharable across tasks. By evaluating the behaviors of prior policies via a hybrid value function architecture, SMEC adaptively aggregates the sharable short-term behaviors of prior policies and the long-term behaviors of the task policy, leading to coordinated decisions. Empirical results on a collection of manipulation and locomotion tasks demonstrate that SMEC outperforms existing methods, and validate the ability of SMEC to leverage related prior policies.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 58 of its 78 references that are in the map. First posted on arXiv 28 May 2023.

Builds on

58 of 78 references.

References · 58

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  6. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  7. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  8. 2017Hindsight Experience Replaycited by 1,136
  9. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  10. 2017Constrained Policy Optimizationcited by 983
  11. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  12. 2017A Distributional Perspective on Reinforcement Learningcited by 765
38 more
  1. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  2. 2019Solving Rubik's Cube with a Robot Handcited by 734
  3. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  4. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 474
  5. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  6. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 369
  7. 2020Planning to Explore via Self-Supervised World Modelscited by 368
  8. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
  9. 2016Variational Intrinsic Controlcited by 267
  10. 2015Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approachcited by 170
  11. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 166
  12. 2018Meta-Gradient Reinforcement Learningcited by 159
  13. 2019Planning with Goal-Conditioned Policiescited by 149
  14. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 143
  15. 2019When to use parametric models in reinforcement learning?cited by 118
  16. 2020Conservative Safety Critics for Explorationcited by 113
  17. 2022Jump-Start Reinforcement Learningcited by 112
  18. 2019MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policiescited by 108
  19. 2023Policy Expansion for Bridging Offline-to-Online Reinforcement Learningcited by 104
  20. 2018Universal Successor Features Approximatorscited by 103
  21. 2018Neural probabilistic motor primitives for humanoid controlcited by 97
  22. 2019Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvementcited by 88
  23. 2021Accelerating Robotic Reinforcement Learning via Parameterized Action Primitivescited by 65
  24. 2022Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progresscited by 65
  25. 2017Hybrid Reward Architecture for Reinforcement Learningcited by 58
  26. 2020Discount Factor as a Regularizer in Reinforcement Learningcited by 45
  27. 2021Learning more skills through optimistic explorationcited by 38
  28. 2022Disentangling Transfer in Continual Reinforcement Learningcited by 33
  29. 2019AC-Teach: A Bayesian Actor-Critic Method for Policy Learning with an Ensemble of Suboptimal Teacherscited by 28
  30. 2023Behavior Contrastive Learning for Unsupervised Skill Discoverycited by 26
  31. 2021Distributional Reinforcement Learning for Multi-Dimensional Reward Functionscited by 22
  32. 2021Beyond Fine-Tuning: Transferring Behavior in Reinforcement Learningcited by 21
  33. 2019Composing Task-Agnostic Policies with Deep Reinforcement Learningcited by 20
  34. 2019MULTIPOLAR: Multi-Source Policy Aggregation for Transfer Reinforcement Learning between Diverse Environmental Dynamicscited by 18
  35. 2018Context-Aware Policy Reusecited by 16
  36. 2022On the Role of Discount Factor in Offline Reinforcement Learningcited by 16
  37. 2019Separating value functions across time-scalescited by 15
  38. 2017An Optimal Online Method of Selecting Source Policies for Reinforcement Learningcited by 13

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Constrained Ensemble Exploration for Unsupervised Skill Discoverycited by 8