# An Adaptive Random Path Selection Approach for Incremental Learning

arXiv: https://arxiv.org/abs/1906.01120 · HTML: https://mlsanity.com/paper/1906.01120/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jun 2019.

## Built on by (most cited first, 17 of 17)

- [DyTox: Transformers for Continual Learning with DYnamic TOken eXpansion](https://mlsanity.com/paper/2111.11326/) (2021)
- [Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Models](https://mlsanity.com/paper/2303.06628/) (2023)
- [A Theoretical Study on Solving Continual Learning](https://mlsanity.com/paper/2211.02633/) (2022)
- [CLIP model is an Efficient Continual Learner](https://mlsanity.com/paper/2210.03114/) (2022)
- [Novel Class Discovery without Forgetting](https://mlsanity.com/paper/2207.10659/) (2022)
- [Energy-based Latent Aligner for Incremental Learning](https://mlsanity.com/paper/2203.14952/) (2022)
- [Learnability and Algorithm for Continual Learning](https://mlsanity.com/paper/2306.12646/) (2023)
- [Continual Learning Based on OOD Detection and Task Masking](https://mlsanity.com/paper/2203.09450/) (2022)
- [Resolving Task Confusion in Dynamic Expansion Architectures for Class Incremental Learning](https://mlsanity.com/paper/2212.14284/) (2022)
- [Spacing Loss for Discovering Novel Categories](https://mlsanity.com/paper/2204.10595/) (2022)
- [D3Former: Debiased Dual Distilled Transformer for Incremental Learning](https://mlsanity.com/paper/2208.00777/) (2022)
- [Balanced Destruction-Reconstruction Dynamics for Memory-replay Class Incremental Learning](https://mlsanity.com/paper/2308.01698/) (2023)
- [Multi-Stage Knowledge Integration of Vision-Language Models for Continual Learning](https://mlsanity.com/paper/2411.06764/) (2024)
- [Reviewing continual learning from the perspective of human-level intelligence](https://mlsanity.com/paper/2111.11964/) (2021)
- [Class Incremental Continual Learning with Self-Organizing Maps and Synthetic Replay](https://mlsanity.com/paper/2508.21240/) (2025)
- [Unsupervised Continual Learning with Growing Self-Organizing Maps and Synthetic Replay](https://mlsanity.com/paper/2608.27662/) (2026)
- [Gated Target Propagation for Compositional Generalization in Continual Learning](https://mlsanity.com/paper/2610.04649/) (2026)

## Abstract (arXiv)

In a conventional supervised learning setting, a machine learning model has access to examples of all object classes that are desired to be recognized during the inference stage. This results in a fixed model that lacks the flexibility to adapt to new learning tasks. In practical settings, learning tasks often arrive in a sequence and the models must continually learn to increment their previously acquired knowledge. Existing incremental learning approaches fall well below the state-of-the-art cumulative models that use all training classes at once. In this paper, we propose a random path selection algorithm, called Adaptive RPS-Net, that progressively chooses optimal paths for the new tasks while encouraging parameter sharing between tasks. We introduce a new network capacity measure that enables us to automatically switch paths if the already used resources are saturated. Since the proposed path-reuse strategy ensures forward knowledge transfer, our approach is efficient and has considerably less computation overhead. As an added novelty, the proposed model integrates knowledge distillation and retrospection along with the path selection strategy to overcome catastrophic forgetting. In order to maintain an equilibrium between previous and newly acquired knowledge, we propose a simple controller to dynamically balance the model plasticity. Through extensive experiments, we demonstrate that the Adaptive RPS-Net method surpasses the state-of-the-art performance for incremental learning and by utilizing parallel computation this method can run in constant time with nearly the same efficiency as a conventional deep convolutional neural network.