# Continual Object Detection via Prototypical Task Correlation Guided Gating Mechanism

arXiv: https://arxiv.org/abs/2205.03055 · HTML: https://mlsanity.com/paper/2205.03055/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 54 references that are in the map.
- First posted on arXiv 6 May 2022.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [Overcoming catastrophic forgetting with hard attention to the task](https://mlsanity.com/paper/1801.01423/) (2018)
- [Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference](https://mlsanity.com/paper/1810.11910/) (2018)
- [Sparse R-CNN: End-to-End Object Detection with Learnable Proposals](https://mlsanity.com/paper/2011.12450/) (2020)
- [Single Path One-Shot Neural Architecture Search with Uniform Sampling](https://mlsanity.com/paper/1904.00420/) (2019)
- [SkipNet: Learning Dynamic Routing in Convolutional Networks](https://mlsanity.com/paper/1711.09485/) (2017)
- [Towards Open World Object Detection](https://mlsanity.com/paper/2103.02603/) (2021)
- [PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search](https://mlsanity.com/paper/1907.05737/) (2019)
- [Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation](https://mlsanity.com/paper/1904.12760/) (2019)
- [Incremental Learning of Object Detectors without Catastrophic Forgetting](https://mlsanity.com/paper/1708.06977/) (2017)
- [Convolutional Networks with Adaptive Inference Graphs](https://mlsanity.com/paper/1711.11503/) (2017)
- [Dynamic Channel Pruning: Feature Boosting and Suppression](https://mlsanity.com/paper/1810.05331/) (2018)
- [Conditional Channel Gated Networks for Task-Aware Continual Learning](https://mlsanity.com/paper/2004.00070/) (2020)
- [Dynamic Slimmable Network](https://mlsanity.com/paper/2103.13258/) (2021)

## Built on by (most cited first, 20 of 20)

- [SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery](https://mlsanity.com/paper/2312.10115/) (2023)
- [SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Model](https://mlsanity.com/paper/2303.05118/) (2023)
- [Continual Detection Transformer for Incremental Object Detection](https://mlsanity.com/paper/2304.03110/) (2023)
- [Weighted Ensemble Models Are Strong Continual Learners](https://mlsanity.com/paper/2312.08977/) (2023)
- [SLCA++: Unleash the Power of Sequential Fine-tuning for Continual Learning with Pre-training](https://mlsanity.com/paper/2408.08295/) (2024)
- [Learning with Style: Continual Semantic Segmentation Across Tasks and Domains](https://mlsanity.com/paper/2210.07016/) (2022)
- [Preventing Catastrophic Forgetting through Memory Networks in Continuous Detection](https://mlsanity.com/paper/2403.14797/) (2024)
- [ATLAS: Adapter-Based Multi-Modal Continual Learning with a Two-Stage Learning Strategy](https://mlsanity.com/paper/2410.10923/) (2024)
- [Unsupervised Recognition of Unknown Objects for Open-World Object Detection](https://mlsanity.com/paper/2308.16527/) (2023)
- [RICO: Two Realistic Benchmarks and an In-Depth Analysis for Incremental Learning in Object Detection](https://mlsanity.com/paper/2508.13878/) (2025)
- [Looking Beyond the Known: Towards a Data Discovery Guided Open-World Object Detection](https://mlsanity.com/paper/2510.00303/) (2025)
- [Out-of-distribution forgetting: vulnerability of continual learning to intra-class distribution shift](https://mlsanity.com/paper/2306.00427/) (2023)
- [C-LoRA: Continual Low-Rank Adaptation for Pre-trained Visual Models](https://mlsanity.com/paper/2502.17920/) (2025)
- [IOR: Inversed Objects Replay for Incremental Object Detection](https://mlsanity.com/paper/2406.04829/) (2024)
- [Incremental Object Detection with Prompt-based Methods](https://mlsanity.com/paper/2508.14599/) (2025)
- [CMFDFormer: Transformer-based Copy-Move Forgery Detection with Continual Learning](https://mlsanity.com/paper/2311.13263/) (2023)
- [Lifelong Domain Adaptive 3D Human Pose Estimation](https://mlsanity.com/paper/2512.23860/) (2025)
- [Knowing the Unknown: Interpretable Open-World Object Detection via Concept Decomposition Model](https://mlsanity.com/paper/2602.20616/) (2026)
- [Enhancing Pre-Trained Model-Based Class-Incremental Learning through Neural Collapse](https://mlsanity.com/paper/2504.18437/) (2025)
- [Hierarchical Dual-Subspace Decoupling for Continual Learning in Vision-Language Models](https://mlsanity.com/paper/2605.07512/) (2026)

## Abstract (arXiv)

Continual learning is a challenging real-world problem for constructing a mature AI system when data are provided in a streaming fashion. Despite recent progress in continual classification, the researches of continual object detection are impeded by the diverse sizes and numbers of objects in each image. Different from previous works that tune the whole network for all tasks, in this work, we present a simple and flexible framework for continual object detection via pRotOtypical taSk corrElaTion guided gaTing mechAnism (ROSETTA). Concretely, a unified framework is shared by all tasks while task-aware gates are introduced to automatically select sub-models for specific tasks. In this way, various knowledge can be successively memorized by storing their corresponding sub-model weights in this system. To make ROSETTA automatically determine which experience is available and useful, a prototypical task correlation guided Gating Diversity Controller(GDC) is introduced to adaptively adjust the diversity of gates for the new task based on class-specific prototypes. GDC module computes class-to-class correlation matrix to depict the cross-task correlation, and hereby activates more exclusive gates for the new task if a significant domain gap is observed. Comprehensive experiments on COCO-VOC, KITTI-Kitchen, class-incremental detection on VOC and sequential learning of four tasks show that ROSETTA yields state-of-the-art performance on both task-based and class-based continual object detection.