On The Transferability of Deep-Q Networks
Not on the map. Why?
Abstract (arXiv)
Transfer Learning (TL) is an efficient machine learning paradigm that allows overcoming some of the hurdles that characterize the successful training of deep neural networks, ranging from long training times to the needs of large datasets. While exploiting TL is a well established and successful training practice in Supervised Learning (SL), its applicability in Deep Reinforcement Learning (DRL) is rarer. In this paper, we study the level of transferability of three different variants of Deep-Q Networks on popular DRL benchmarks as well as on a set of novel, carefully designed control tasks. Our results show that transferring neural networks in a DRL context can be particularly challenging and is a process which in most cases results in negative transfer. In the attempt of understanding why Deep-Q Networks transfer so poorly, we gain novel insights into the training dynamics that characterizes this family of algorithms.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 33 of its 49 references that are in the map. First posted on arXiv 6 Oct 2021.
Builds on
33 of 49 references.
References · 33
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2015Continuous control with deep reinforcement learningcited by 3,279
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
- 2015Trust Region Policy Optimizationcited by 2,503
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
- 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
- 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
- 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
- 2018World Modelscited by 1,444
- 2016Progressive Neural Networkscited by 1,332
21 more
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
- 2019A Comprehensive Survey on Transfer Learningcited by 1,154
- 2020Mastering Atari with Discrete World Modelscited by 990
- 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
- 2017Deep Reinforcement Learning that Matterscited by 727
- 2019Model-Based Reinforcement Learning for Ataricited by 495
- 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
- 2014Recurrent Models of Visual Attentioncited by 344
- 2014CNN Features off-the-shelf: an Astounding Baseline for Recognitioncited by 316
- 2018An Introduction to Deep Reinforcement Learningcited by 201
- 2016What makes ImageNet good for transfer learning?cited by 192
- 2016Sample Efficient Actor-Critic with Experience Replaycited by 185
- 2015Actor-Mimic: Deep Multitask and Transfer Reinforcement Learningcited by 176
- 2018Generalization and Regularization in DQNcited by 117
- 2020Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning Researchcited by 78
- 2021Factors of Influence for Transfer Learning across Diverse Appearance Domains and Task Typescited by 54
- 2019A Model-based Approach for Sample-efficient Multi-task Reinforcement Learningcited by 12
- 2021Improving Computational Efficiency in Visual Reinforcement Learning via Stored Embeddingscited by 12
- 2021Fractional Transfer Learning for Deep Model-Based Reinforcement Learningcited by 6
- 2018Deep Quality-Value (DQV) Learningcited by 5
- 2020How Transferable are the Representations Learned by Deep Q Agents?cited by 4
Built on by
Citing papers per year: 2022: 1; 2023: 1; 2024: 0; 2025: 1.
3 papers.