Efficient Open-world Reinforcement Learning via Knowledge Distillation and Autonomous Rule Discovery
Not on the map. Why?
Abstract (arXiv)
Deep reinforcement learning suffers from catastrophic forgetting and sample inefficiency making it less applicable to the ever-changing real world. However, the ability to use previously learned knowledge is essential for AI agents to quickly adapt to novelties. Often, certain spatial information observed by the agent in the previous interactions can be leveraged to infer task-specific rules. Inferred rules can then help the agent to avoid potentially dangerous situations in the previously unseen states and guide the learning process increasing agent's novelty adaptation speed. In this work, we propose a general framework that is applicable to deep reinforcement learning agents. Our framework provides the agent with an autonomous way to discover the task-specific rules in the novel environments and self-supervise it's learning. We provide a rule-driven deep Q-learning agent (RDQ) as one possible implementation of that framework. We show that RDQ successfully extracts task-specific rules as it interacts with the world and uses them to drastically increase its learning efficiency. In our experiments, we show that the RDQ agent is significantly more resilient to the novelties than the baseline agents, and is able to detect and adapt to novel situations faster.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 51 of its 116 references that are in the map. First posted on arXiv 24 Nov 2023.
Builds on
51 of 116 references.
References · 51
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
- 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
- 2016Concrete Problems in AI Safetycited by 1,582
- 2016Generative Adversarial Imitation Learningcited by 1,576
- 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
- 2017Constrained Policy Optimizationcited by 928
- 2015End-to-End Training of Deep Visuomotor Policiescited by 847
- 2014Deep Learning in Neural Networks: An Overviewcited by 765
- 2017One pixel attack for fooling deep neural networkscited by 579
38 more
- 2017Safe Reinforcement Learning via Shieldingcited by 427
- 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
- 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 350
- 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
- 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 278
- 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 268
- 2017Reverse Curriculum Generation for Reinforcement Learningcited by 185
- 2016ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learningcited by 178
- 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
- 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 164
- 2019Unsupervised State Representation Learning in Ataricited by 144
- 2015Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction modelcited by 140
- 2017Trial without Error: Towards Safe Reinforcement Learning via Human Interventioncited by 102
- 2016A Deep Hierarchical Approach to Lifelong Learning in Minecraftcited by 90
- 2020Learning programs by learning from failurescited by 73
- 2018Safe Reinforcement Learning with Model Uncertainty Estimatescited by 70
- 2021A Survey on Interpretable Reinforcement Learningcited by 70
- 2016Scalable Bayesian Rule Listscited by 64
- 2017Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physicscited by 57
- 2016Safe Policy Improvement by Minimizing Robust Baseline Regretcited by 51
- 2021Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Drivingcited by 46
- 2018Learning to Teach in Cooperative Multiagent Reinforcement Learningcited by 45
- 2014Safe Exploration of State and Action Spaces in Reinforcement Learningcited by 37
- 2019Understanding Multi-Step Deep Reinforcement Learning: A Systematic Study of the DQN Targetcited by 31
- 2021Neuro-Symbolic Reinforcement Learning with First-Order Logiccited by 30
- 2016Strategic Attentive Writer for Learning Macro-Actionscited by 29
- 2017Data-Efficient Policy Evaluation Through Behavior Policy Searchcited by 25
- 2018StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer Learningcited by 25
- 2015Attend, Adapt and Transfer: Attentive Deep Architecture for Adaptive Transfer from multiple sources in the same domaincited by 19
- 2018Context-Aware Policy Reusecited by 15
- 2017An Optimal Online Method of Selecting Source Policies for Reinforcement Learningcited by 13
- 2022RAPid-Learn: A Framework for Learning to Recover for Handling Novelties in Open-World Environmentscited by 13
- 2019A Model-based Approach for Sample-efficient Multi-task Reinforcement Learningcited by 12
- 2020Incorporating Relational Background Knowledge into Reinforcement Learning via Differentiable Inductive Logic Programmingcited by 12
- 2017RAIL: Risk-Averse Imitation Learningcited by 9
- 2021Interpretable Model-based Hierarchical Reinforcement Learning using Inductive Logic Programmingcited by 8
- 2021KnowRU: Knowledge Reusing via Knowledge Distillation in Multi-agent Reinforcement Learningcited by 6
- 2017Learning to Navigate Cloth using Hapticscited by 4
50 most cited of 51 All 51 (JSON)
Built on by
Citing papers per year: 2024: 1.
1 paper.