# Sylph: A Hypernetwork Framework for Incremental Few-shot Object Detection

arXiv: https://arxiv.org/abs/2203.13903 · HTML: https://mlsanity.com/paper/2203.13903/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 72 references that are in the map.
- First posted on arXiv 25 Mar 2022.

## Builds on (most cited first, 30 of 53)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [CenterNet: Keypoint Triplets for Object Detection](https://mlsanity.com/paper/1904.08189/) (2019)
- [Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation](https://mlsanity.com/paper/2012.07177/) (2020)
- [Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks](https://mlsanity.com/paper/1602.07868/) (2016)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [R-FCN: Object Detection via Region-based Fully Convolutional Networks](https://mlsanity.com/paper/1605.06409/) (2016)
- [Deep Learning for Generic Object Detection: A Survey](https://mlsanity.com/paper/1809.02165/) (2018)
- [Dynamic Few-Shot Visual Learning without Forgetting](https://mlsanity.com/paper/1804.09458/) (2018)

## Built on by (most cited first, 23 of 23)

- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [A Brief Review of Hypernetworks in Deep Learning](https://mlsanity.com/paper/2306.06955/) (2023)
- [A Survey on Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2304.08130/) (2023)
- [EgoLoc: Revisiting 3D Object Localization from Egocentric Videos with Visual Queries](https://mlsanity.com/paper/2212.06969/) (2022)
- [Few-shot Class-incremental Learning for Classification and Object Detection: A Survey](https://mlsanity.com/paper/2308.06764/) (2023)
- [Where is my Wallet? Modeling Object Proposal Sets for Egocentric Visual Query Localization](https://mlsanity.com/paper/2211.10528/) (2022)
- [Few-shot Learning with Noisy Labels](https://mlsanity.com/paper/2204.05494/) (2022)
- [Multimodal Parameter-Efficient Few-Shot Class Incremental Learning](https://mlsanity.com/paper/2303.04751/) (2023)
- [Zero-shot Generalizable Incremental Learning for Vision-Language Object Detection](https://mlsanity.com/paper/2403.01680/) (2024)
- [When Meta-Learning Meets Online and Continual Learning: A Survey](https://mlsanity.com/paper/2311.05241/) (2023)
- [Recent Few-Shot Object Detection Algorithms: A Survey with Performance Comparison](https://mlsanity.com/paper/2203.14205/) (2022)
- [Identification of Novel Classes for Improving Few-Shot Object Detection](https://mlsanity.com/paper/2303.10422/) (2023)
- [Improved Region Proposal Network for Enhanced Few-Shot Object Detection](https://mlsanity.com/paper/2308.07535/) (2023)
- [Few-shot Object Detection with Refined Contrastive Learning](https://mlsanity.com/paper/2211.13495/) (2022)
- [Meta-tuning Loss Functions and Data Augmentation for Few-shot Object Detection](https://mlsanity.com/paper/2304.12161/) (2023)
- [Continual HyperTransformer: A Meta-Learner for Continual Few-Shot Learning](https://mlsanity.com/paper/2301.04584/) (2023)
- [Incremental Few-Shot Object Detection via Simple Fine-Tuning Approach](https://mlsanity.com/paper/2302.09779/) (2023)
- [Few-Shot Incremental 3D Object Detection in Dynamic Indoor Environments](https://mlsanity.com/paper/2604.07997/) (2026)
- [CMFDFormer: Transformer-based Copy-Move Forgery Detection with Continual Learning](https://mlsanity.com/paper/2311.13263/) (2023)
- [Evaluating the Energy Efficiency of Few-Shot Learning for Object Detection in Industrial Settings](https://mlsanity.com/paper/2403.06631/) (2024)
- [Learning to Generate Parameters of ConvNets for Unseen Image Data](https://mlsanity.com/paper/2310.11862/) (2023)
- [InfRS: Incremental Few-Shot Object Detection in Remote Sensing Images](https://mlsanity.com/paper/2405.11293/) (2024)
- [ZeBROD: Zero-Retraining Based Recognition and Object Detection Framework](https://mlsanity.com/paper/2512.04888/) (2025)

## Abstract (arXiv)

We study the challenging incremental few-shot object detection (iFSD) setting. Recently, hypernetwork-based approaches have been studied in the context of continuous and finetune-free iFSD with limited success. We take a closer look at important design choices of such methods, leading to several key improvements and resulting in a more accurate and flexible framework, which we call Sylph. In particular, we demonstrate the effectiveness of decoupling object classification from localization by leveraging a base detector that is pretrained for class-agnostic localization on a large-scale dataset. Contrary to what previous results have suggested, we show that with a carefully designed class-conditional hypernetwork, finetune-free iFSD can be highly effective, especially when a large number of base categories with abundant data are available for meta-training, almost approaching alternatives that undergo test-time-training. This result is even more significant considering its many practical advantages: (1) incrementally learning new classes in sequence without additional training, (2) detecting both novel and seen classes in a single pass, and (3) no forgetting of previously seen classes. We benchmark our model on both COCO and LVIS, reporting as high as 17% AP on the long-tail rare classes on LVIS, indicating the promise of hypernetwork-based iFSD.