mlsanityData to 7 Oct 2026

Online Learning via Memory: Retrieval-Augmented Detector Adaptation

arXiv 2409.10716 ↗First posted 16 Sep 2024cs.CV cs.IR cs.LGcited by 0

On the map: Retrieval-augmented generation → Multimodal RAG

Abstract (arXiv)

This paper presents a novel way of online adapting any off-the-shelf object detection model to a novel domain without retraining the detector model. Inspired by how humans quickly learn knowledge of a new subject (e.g., memorization), we allow the detector to look up similar object concepts from memory during test time. This is achieved through a retrieval augmented classification (RAC) module together with a memory bank that can be flexibly updated with new domain knowledge. We experimented with various off-the-shelf open-set detector and close-set detectors. With only a tiny memory bank (e.g., 10 images per category) and being training-free, our online learning method could significantly outperform baselines in adapting a detector to novel domains.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 14 of its 14 references that are in the map. First posted on arXiv 16 Sep 2024.

Builds on

14 of 14 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Retrieval-Augmented Open-Vocabulary Object Detection8 Apr
  2. 2023Learning Customized Visual Models with Retrieval-Augmented Knowledge17 Jan
  3. 2022Retrieval Augmented Classification for Long-Tail Visual Recognition22 Feb

Other references · 11

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  7. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  8. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  9. 2021Grounded Language-Image Pre-trainingcited by 1,147
  10. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  11. 2017DOTA: A Large-scale Dataset for Object Detection in Aerial Imagescited by 622

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Improving Image Recognition by Retrieving from Web-Scale Image-Text Data
  2. 2025Retrieval-Augmented Memory for Online Learning
  3. 2023Efficient Adaptive Human-Object Interaction Detection with Concept-guided Memory
  4. 2025Memory-enhanced Retrieval Augmentation for Long Video Understanding
  5. 2024Understanding Retrieval-Augmented Task Adaptation for Vision-Language Models
  6. 2026Reward-Guided Semantic Evolution for Test-time Adaptive Object Detection
  7. 2025Bayesian Test-time Adaptation for Object Recognition and Detection with Vision-language Models
  8. 2024D'ej`a Vu Memorization in Vision-Language Models