mlsanityData to 13 Oct 2026

Flexible Attention-Based Multi-Policy Fusion for Efficient Deep Reinforcement Learning

arXiv 2210.03729 ↗First posted 7 Oct 2022cs.LG cs.AI cs.ROcited by 5

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning (RL) agents have long sought to approach the efficiency of human learning. Humans are great observers who can learn by aggregating external knowledge from various sources, including observations from others' policies of attempting a task. Prior studies in RL have incorporated external knowledge policies to help agents improve sample efficiency. However, it remains non-trivial to perform arbitrary combinations and replacements of those policies, an essential feature for generalization and transferability. In this work, we present Knowledge-Grounded RL (KGRL), an RL paradigm fusing multiple knowledge policies and aiming for human-like efficiency and flexibility. We propose a new actor architecture for KGRL, Knowledge-Inclusive Attention Network (KIAN), which allows free knowledge rearrangement due to embedding-based attentive action prediction. KIAN also addresses entropy imbalance, a problem arising in maximum entropy KGRL that hinders an agent from efficiently exploring the environment, through a new design of policy distributions. The experimental results demonstrate that KIAN outperforms alternative methods incorporating external knowledge policies and achieves efficient and flexible learning. Our implementation is available at https://github.com/Pascalson/KGRL.git

Key facts. Cited by 5 papers in the map (data through 13 Oct 2026). Builds on 28 of its 48 references that are in the map. First posted on arXiv 7 Oct 2022.

Builds on

28 of 48 references.

References · 28

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,223
  2. 2017Proximal Policy Optimization Algorithmscited by 14,592
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,656
  4. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,726
  5. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 731
  6. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 687
  7. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 660
  8. 2016The Option-Critic Architecturecited by 439
  9. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 325
  10. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 304
  11. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 282
  12. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 279
16 more
  1. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 257
  2. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  3. 2019Language as an Abstraction for Hierarchical Deep Reinforcement Learningcited by 138
  4. 2020Provably Efficient Safe Exploration via Primal-Dual Policy Optimizationcited by 132
  5. 2018Near-Optimal Representation Learning for Hierarchical Reinforcement Learningcited by 108
  6. 2022Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progresscited by 64
  7. 2014Safe Exploration of State and Action Spaces in Reinforcement Learningcited by 37
  8. 2018Reinforced Imitation: Sample Efficient Deep Reinforcement Learning for Map-less Navigation by Leveraging Prior Demonstrationscited by 35
  9. 2021Unsupervised Skill Discovery with Bottleneck Option Learningcited by 28
  10. 2017Deep Q-learning from Demonstrationscited by 27
  11. 2019Composing Task-Agnostic Policies with Deep Reinforcement Learningcited by 20
  12. 2015Attend, Adapt and Transfer: Attentive Deep Architecture for Adaptive Transfer from multiple sources in the same domaincited by 19
  13. 2017A Low-Cost Ethics Shaping Approach for Designing Reinforcement Learning Agentscited by 18
  14. 2020Reinforcement Learning Under Moral Uncertaintycited by 16
  15. 2020KoGuN: Accelerating Deep Reinforcement Learning via Integrating Human Suboptimal Knowledgecited by 13
  16. 2021Toward Robust Long Range Policy Transfercited by 6

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 3; 2025: 1.

5 papers.

  1. 2024Improving Sample Efficiency of Reinforcement Learning with Background Knowledge from Large Language Modelscited by 4
  2. 2023Heterogeneous Knowledge for Augmented Modular Reinforcement Learningcited by 1
  3. 2025CARoL: Context-aware Adaptation for Robot Learningcited by 1
  4. 2024SurgIRL: Towards Life-Long Learning for Surgical Automation by Incremental Reinforcement Learningcited by 0
  5. 2024Conceptual Belief-Informed Reinforcement Learningcited by 0