# Incremental Learning of Object Detectors without Catastrophic Forgetting

arXiv: https://arxiv.org/abs/1708.06977 · HTML: https://mlsanity.com/paper/1708.06977/

## Key facts

- Cited by 208 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Aug 2017.

## Built on by (most cited first, 30 of 208)

- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Towards Open World Object Detection](https://mlsanity.com/paper/2103.02603/) (2021)
- [Online Continual Learning in Image Classification: An Empirical Survey](https://mlsanity.com/paper/2101.10423/) (2021)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Model](https://mlsanity.com/paper/2303.05118/) (2023)
- [OW-DETR: Open-world Detection Transformer](https://mlsanity.com/paper/2112.01513/) (2021)
- [Federated Class-Incremental Learning](https://mlsanity.com/paper/2203.11473/) (2022)
- [AutoNovel: Automatically Discovering and Learning Novel Visual Categories](https://mlsanity.com/paper/2106.15252/) (2021)
- [Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2103.04059/) (2021)
- [Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations](https://mlsanity.com/paper/2103.06342/) (2021)
- [RECALL: Replay-based Continual Learning in Semantic Segmentation](https://mlsanity.com/paper/2108.03673/) (2021)
- [Overcoming Catastrophic Forgetting in Incremental Object Detection via Elastic Response Distillation](https://mlsanity.com/paper/2204.02136/) (2022)
- [Continual Detection Transformer for Incremental Object Detection](https://mlsanity.com/paper/2304.03110/) (2023)
- [Lifelong Person Re-Identification via Adaptive Knowledge Accumulation](https://mlsanity.com/paper/2103.12462/) (2021)
- [PROB: Probabilistic Objectness for Open World Object Detection](https://mlsanity.com/paper/2212.01424/) (2022)
- [Recent Advances of Continual Learning in Computer Vision: An Overview](https://mlsanity.com/paper/2109.11369/) (2021)
- [Factors of Influence for Transfer Learning across Diverse Appearance Domains and Task Types](https://mlsanity.com/paper/2103.13318/) (2021)
- [Self-Supervised Training Enhances Online Continual Learning](https://mlsanity.com/paper/2103.14010/) (2021)
- [No One Left Behind: Real-World Federated Class-Incremental Learning](https://mlsanity.com/paper/2302.00903/) (2023)
- [Layerwise Optimization by Gradient Decomposition for Continual Learning](https://mlsanity.com/paper/2105.07561/) (2021)
- [A Survey on Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2304.08130/) (2023)
- [Class-Incremental Learning for Action Recognition in Videos](https://mlsanity.com/paper/2203.13611/) (2022)
- [SDDGR: Stable Diffusion-based Deep Generative Replay for Class Incremental Object Detection](https://mlsanity.com/paper/2402.17323/) (2024)
- [Memory Efficient Continual Learning with Transformers](https://mlsanity.com/paper/2203.04640/) (2022)
- [On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning](https://mlsanity.com/paper/2210.06443/) (2022)
- [Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation](https://mlsanity.com/paper/2203.14098/) (2022)
- [Energy-based Latent Aligner for Incremental Learning](https://mlsanity.com/paper/2203.14952/) (2022)
- [Multi-View Correlation Distillation for Incremental Object Detection](https://mlsanity.com/paper/2107.01787/) (2021)
- [CLAD: A realistic Continual Learning benchmark for Autonomous Driving](https://mlsanity.com/paper/2210.03482/) (2022)
- [Wanderlust: Online Continual Object Detection in the Real World](https://mlsanity.com/paper/2108.11005/) (2021)

## Abstract (arXiv)

Despite their success for object detection, convolutional neural networks are ill-equipped for incremental learning, i.e., adapting the original model trained on a set of classes to additionally detect objects of new classes, in the absence of the initial training data. They suffer from "catastrophic forgetting" - an abrupt degradation of performance on the original set of classes, when the training objective is adapted to the new classes. We present a method to address this issue, and learn object detectors incrementally, when neither the original training data nor annotations for the original classes in the new training set are available. The core of our proposed solution is a loss function to balance the interplay between predictions on the new classes and a new distillation loss which minimizes the discrepancy between responses for old classes from the original and the updated networks. This incremental learning can be performed multiple times, for a new set of classes in each step, with a moderate drop in performance compared to the baseline network trained on the ensemble of data. We present object detection results on the PASCAL VOC 2007 and COCO datasets, along with a detailed empirical analysis of the approach.