# Modeling Missing Annotations for Incremental Learning in Object Detection

arXiv: https://arxiv.org/abs/2204.08766 · HTML: https://mlsanity.com/paper/2204.08766/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 43 of its 62 references that are in the map.
- First posted on arXiv 19 Apr 2022.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 30 of 43)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence](https://mlsanity.com/paper/1801.10112/) (2018)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [DER: Dynamically Expandable Representation for Class Incremental Learning](https://mlsanity.com/paper/2103.16788/) (2021)
- [Towards Open World Object Detection](https://mlsanity.com/paper/2103.02603/) (2021)
- [DyTox: Transformers for Continual Learning with DYnamic TOken eXpansion](https://mlsanity.com/paper/2111.11326/) (2021)
- [Learning without Memorizing](https://mlsanity.com/paper/1811.08051/) (2018)
- [Incremental Learning of Object Detectors without Catastrophic Forgetting](https://mlsanity.com/paper/1708.06977/) (2017)
- [Modeling the Background for Incremental Learning in Semantic Segmentation](https://mlsanity.com/paper/2002.00718/) (2020)
- [Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learning](https://mlsanity.com/paper/1904.03137/) (2019)
- [PLOP: Learning without Forgetting for Continual Semantic Segmentation](https://mlsanity.com/paper/2011.11390/) (2020)
- [Incremental Learning Techniques for Semantic Segmentation](https://mlsanity.com/paper/1907.13372/) (2019)

## Built on by (most cited first, 23 of 23)

- [Augmented Box Replay: Overcoming Foreground Shift for Incremental Object Detection](https://mlsanity.com/paper/2307.12427/) (2023)
- [CoMFormer: Continual Learning in Semantic and Panoptic Segmentation](https://mlsanity.com/paper/2211.13999/) (2022)
- [VLM-PL: Advanced Pseudo Labeling Approach for Class Incremental Object Detection via Vision-Language Model](https://mlsanity.com/paper/2403.05346/) (2024)
- [Online Continual Learning For Interactive Instruction Following Agents](https://mlsanity.com/paper/2403.07548/) (2024)
- [ECLIPSE: Efficient Continual Learning in Panoptic Segmentation with Visual Prompt Tuning](https://mlsanity.com/paper/2403.20126/) (2024)
- [Bridge Past and Future: Overcoming Information Asymmetry in Incremental Object Detection](https://mlsanity.com/paper/2407.11499/) (2024)
- [Demystifying Catastrophic Forgetting in Two-Stage Incremental Object Detector](https://mlsanity.com/paper/2502.05540/) (2025)
- [CoMBO: Conflict Mitigation via Branched Optimization for Class Incremental Segmentation](https://mlsanity.com/paper/2504.04156/) (2025)
- [iPINNs: Incremental learning for Physics-informed neural networks](https://mlsanity.com/paper/2304.04854/) (2023)
- [Elastic Weight Consolidation Done Right for Continual Learning](https://mlsanity.com/paper/2603.18596/) (2026)
- [Class-Incremental Learning of Plant and Disease Detection: Growing Branches with Knowledge Distillation](https://mlsanity.com/paper/2304.06619/) (2023)
- [Dynamic Y-KD: A Hybrid Approach to Continual Instance Segmentation](https://mlsanity.com/paper/2303.06015/) (2023)
- [Tackling the Incomplete Annotation Issue in Universal Lesion Detection Task By Exploratory Training](https://mlsanity.com/paper/2309.13306/) (2023)
- [Teach YOLO to Remember: A Self-Distillation Approach for Continual Object Detection](https://mlsanity.com/paper/2503.04688/) (2025)
- [DuET: Dual Incremental Object Detection via Exemplar-Free Task Arithmetic](https://mlsanity.com/paper/2506.21260/) (2025)
- [Learning on the Fly: Replay-Based Continual Object Perception for Indoor Drones](https://mlsanity.com/paper/2602.13440/) (2026)
- [EW-DETR: Evolving World Object Detection via Incremental Low-Rank DEtection TRansformer](https://mlsanity.com/paper/2602.20985/) (2026)
- [STAR-IOD: Scale-decoupled Topology Alignment with Pseudo-label Refinement for Remote Sensing Incremental Object Detection](https://mlsanity.com/paper/2605.20738/) (2026)
- [Focus, Align, and Sustain: Counteracting Gradient Dilution in Incremental Object Detection](https://mlsanity.com/paper/2606.15253/) (2026)
- [CASA: Class-Agnostic Shared Attributes in Vision-Language Models for Efficient Incremental Object Detection](https://mlsanity.com/paper/2410.05804/) (2024)
- [Exploiting Stability-Plasticity Asymmetry in Pretrained Detectors for Incremental Object Detection](https://mlsanity.com/paper/2504.10214/) (2025)
- [Parameterized Prompt for Incremental Object Detection](https://mlsanity.com/paper/2510.27316/) (2025)
- [Beyond Prompt Degradation: Prototype-guided Dual-pool Prompting for Incremental Object Detection](https://mlsanity.com/paper/2603.02286/) (2026)

## Abstract (arXiv)

Despite the recent advances in the field of object detection, common architectures are still ill-suited to incrementally detect new categories over time. They are vulnerable to catastrophic forgetting: they forget what has been already learned while updating their parameters in absence of the original training data. Previous works extended standard classification methods in the object detection task, mainly adopting the knowledge distillation framework. However, we argue that object detection introduces an additional problem, which has been overlooked. While objects belonging to new classes are learned thanks to their annotations, if no supervision is provided for other objects that may still be present in the input, the model learns to associate them to background regions. We propose to handle these missing annotations by revisiting the standard knowledge distillation framework. Our approach outperforms current state-of-the-art methods in every setting of the Pascal-VOC dataset. We further propose an extension to instance segmentation, outperforming the other baselines. Code can be found here: https://github.com/fcdl94/MMA