# Less-forgetful Learning for Domain Expansion in Deep Neural Networks

arXiv: https://arxiv.org/abs/1711.05959 · HTML: https://mlsanity.com/paper/1711.05959/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Nov 2017.

## Built on by (most cited first, 27 of 27)

- [Online Continual Learning in Image Classification: An Empirical Survey](https://mlsanity.com/paper/2101.10423/) (2021)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [RL's Razor: Why Online Reinforcement Learning Forgets Less](https://mlsanity.com/paper/2509.04259/) (2025)
- [Multi-View Correlation Distillation for Incremental Object Detection](https://mlsanity.com/paper/2107.01787/) (2021)
- [Continual Learning via Sequential Function-Space Variational Inference](https://mlsanity.com/paper/2312.17210/) (2023)
- [Class-Balanced Distillation for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2104.05279/) (2021)
- [Upweighting Easy Samples in Fine-Tuning Mitigates Forgetting](https://mlsanity.com/paper/2502.02797/) (2025)
- [Towards Unsupervised Open World Semantic Segmentation](https://mlsanity.com/paper/2201.01073/) (2022)
- [Continual Learning on Graphs: A Survey](https://mlsanity.com/paper/2402.06330/) (2024)
- [Few-Shot Domain Expansion for Face Anti-Spoofing](https://mlsanity.com/paper/2106.14162/) (2021)
- [Improving Replay Sample Selection and Storage for Less Forgetting in Continual Learning](https://mlsanity.com/paper/2308.01895/) (2023)
- [Mitigating Sexual Content Generation via Embedding Distortion in Text-conditioned Diffusion Models](https://mlsanity.com/paper/2501.18877/) (2025)
- [CL2R: Compatible Lifelong Learning Representations](https://mlsanity.com/paper/2211.09032/) (2022)
- [RTRA: Rapid Training of Regularization-based Approaches in Continual Learning](https://mlsanity.com/paper/2312.09361/) (2023)
- [Lifetime policy reuse and the importance of task capacity](https://mlsanity.com/paper/2106.01741/) (2021)
- [Detecting and Learning the Unknown in Semantic Segmentation](https://mlsanity.com/paper/2202.08700/) (2022)
- [Exploring the Intersection between Neural Architecture Search and Continual Learning](https://mlsanity.com/paper/2206.05625/) (2022)
- [ARC: Adversarially Robust Control Policies for Autonomous Vehicles](https://mlsanity.com/paper/2107.04487/) (2021)
- [Learning an evolved mixture model for task-free continual learning](https://mlsanity.com/paper/2207.05080/) (2022)
- [FDCNet: Feature Drift Compensation Network for Class-Incremental Weakly Supervised Object Localization](https://mlsanity.com/paper/2309.09122/) (2023)
- [Center Loss Regularization for Continual Learning](https://mlsanity.com/paper/2110.11314/) (2021)
- [SequeL: A Continual Learning Library in PyTorch and JAX](https://mlsanity.com/paper/2304.10857/) (2023)
- [Sample Compression for Self Certified Continual Learning](https://mlsanity.com/paper/2503.10503/) (2025)
- [Self-Supervised Knowledge Transfer via Loosely Supervised Auxiliary Tasks](https://mlsanity.com/paper/2110.12696/) (2021)
- [Propheter: Prophetic Teacher Guided Long-Tailed Distribution Learning](https://mlsanity.com/paper/2304.04135/) (2023)
- [Continual Cross-Dataset Adaptation in Road Surface Classification](https://mlsanity.com/paper/2309.02210/) (2023)
- [Geometry of Forgetting: Representation Flux in Continual Learning](https://mlsanity.com/paper/2608.15854/) (2026)

## Abstract (arXiv)

Expanding the domain that deep neural network has already learned without accessing old domain data is a challenging task because deep neural networks forget previously learned information when learning new data from a new domain. In this paper, we propose a less-forgetful learning method for the domain expansion scenario. While existing domain adaptation techniques solely focused on adapting to new domains, the proposed technique focuses on working well with both old and new domains without needing to know whether the input is from the old or new domain. First, we present two naive approaches which will be problematic, then we provide a new method using two proposed properties for less-forgetful learning. Finally, we prove the effectiveness of our method through experiments on image classification tasks. All datasets used in the paper, will be released on our website for someone's follow-up study.