mlsanityData to 7 Oct 2026

Progressive Neural Networks

arXiv 1606.04671 ↗First posted 15 Jun 2016cs.LGcited by 1,332

Not on the map. Why?

Cited by 24 in Mixture-of-experts, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Learning to solve complex sequences of tasks--while both leveraging transfer and avoiding catastrophic forgetting--remains a key obstacle to achieving human-level intelligence. The progressive networks approach represents a step forward in this direction: they are immune to forgetting and can leverage prior knowledge via lateral connections to previously learned features. We evaluate this architecture extensively on a wide variety of reinforcement learning tasks (Atari and 3D maze games), and show that it outperforms common baselines based on pretraining and finetuning. Using a novel sensitivity measure, we demonstrate that transfer occurs at both low-level sensory and high-level control layers of the learned policy.

Key facts. Cited by 1,332 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Jun 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 200; 2022: 227; 2023: 262; 2024: 243; 2025: 190; 2026: 209.

1,332 papers.

Sort:

By area:

  1. 2022Generative Adversarial Networkscited by 5,603
  2. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  3. 2021Learning to Prompt for Continual Learningcited by 938
  4. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  5. 2022A Generalist Agentcited by 802
  6. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 635
  7. 2021iMAP: Implicit Mapping and Positioning in Real-Timecited by 476
  8. 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 446
  9. 2021Gradient Projection Memory for Continual Learningcited by 338
  10. 2021Towards Open World Object Detectioncited by 297
  11. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
  12. 2021Co2L: Contrastive Continual Learningcited by 243
38 more
  1. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 242
  2. 2022FOSTER: Feature Boosting and Compression for Class-Incremental Learningcited by 228
  3. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
  4. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 226
  5. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  6. 2021Efficiently Identifying Task Groupings for Multi-Task Learningcited by 218
  7. 2023Class-Incremental Learning: A Surveycited by 209
  8. 2023Perpetual Humanoid Control for Real-time Simulated Avatarscited by 198
  9. 2023Transformer-Patcher: One Mistake worth One Neuroncited by 196
  10. 2023Progressive Prompts: Continual Learning for Language Modelscited by 195
  11. 2024InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learningcited by 189
  12. 2022Patching open-vocabulary models by interpolating weightscited by 187
  13. 2023SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Modelcited by 178
  14. 2023Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Modelscited by 175
  15. 2021Towards Continual Knowledge Learning of Language Modelscited by 172
  16. 2022A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learningcited by 152
  17. 2021Self-Supervised Models are Continual Learnerscited by 149
  18. 2023Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRAcited by 138
  19. 2021An Empirical Investigation of the Role of Pre-training in Lifelong Learningcited by 134
  20. 2021Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learningcited by 130
  21. 2021DEMix Layers: Disentangling Domains for Modular Language Modelingcited by 130
  22. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  23. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 124
  24. 2022Continual Learning and Private Unlearningcited by 124
  25. 2023A Unified Continual Learning Framework with General Parameter-Efficient Tuningcited by 124
  26. 2022Class-Incremental Continual Learning into the eXtended DER-versecited by 121
  27. 2022Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learningcited by 118
  28. 2021Online Coreset Selection for Rehearsal-based Continual Learningcited by 114
  29. 2022Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidationcited by 113
  30. 2021Continual World: A Robotic Benchmark For Continual Reinforcement Learningcited by 112
  31. 2022Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning Systemcited by 110
  32. 2024Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLMcited by 106
  33. 2022Remember the Past: Distilling Datasets into Addressable Memories for Neural Networkscited by 104
  34. 2021LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5cited by 103
  35. 2021Representational Continuity for Unsupervised Continual Learningcited by 95
  36. 2021Achieving Forgetting Prevention and Knowledge Transfer in Continual Learningcited by 91
  37. 2021Continual Learning for Text Classification with Information Disentanglement Based Regularizationcited by 86
  38. 2021Continual Learning in the Teacher-Student Setup: Impact of Task Similaritycited by 86

50 most cited of 1,332 All 1,332 (JSON)