mlsanityData to 7 Oct 2026

CASA: Class-Agnostic Shared Attributes in Vision-Language Models for Efficient Incremental Object Detection

arXiv 2410.05804 ↗First posted 8 Oct 2024cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Driving VLMs and open-vocabulary detection

Abstract (arXiv)

Incremental object detection is fundamentally challenged by catastrophic forgetting. A major factor contributing to this issue is background shift, where background categories in sequential tasks may overlap with either previously learned or future unseen classes. To address this, we propose a novel method called Class-Agnostic Shared Attribute Base (CASA) that encourages the model to learn category-agnostic attributes shared across incremental classes. Our approach leverages an LLM to generate candidate textual attributes, selects the most relevant ones based on the current training data, and records their importance in an assignment matrix. For subsequent tasks, the retained attributes are frozen, and new attributes are selected from the remaining candidates, ensuring both knowledge retention and adaptability. Extensive experiments on the COCO dataset demonstrate the state-of-the-art performance of our method.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 50 of its 64 references that are in the map. First posted on arXiv 8 Oct 2024.

Builds on

50 of 64 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024VLM-PL: Advanced Pseudo Labeling Approach for Class Incremental Object Detection via Vision-Language Model8 Mar

Other references · 49

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2017Mask R-CNNcited by 7,259
  7. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  8. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  9. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  10. 2015Fast R-CNNcited by 2,930
  11. 2018YOLOv3: An Incremental Improvementcited by 2,652
  12. 2016Learning without Forgettingcited by 2,465
37 more
  1. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  2. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  3. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  4. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  5. 2017Adversarial Discriminative Domain Adaptationcited by 1,309
  6. 2021Learning to Prompt for Continual Learningcited by 938
  7. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  8. 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
  9. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 635
  10. 2019Object Detection in 20 Years: A Surveycited by 528
  11. 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 446
  12. 2022Visual Classification via Description from Large Language Modelscited by 331
  13. 2022S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learningcited by 246
  14. 2017Incremental Learning of Object Detectors without Catastrophic Forgettingcited by 208
  15. 2020Modeling the Background for Incremental Learning in Semantic Segmentationcited by 189
  16. 2023SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Modelcited by 178
  17. 2023RanPAC: Random Projections and Pre-trained Models for Continual Learningcited by 162
  18. 2024Continual Learning with Pre-Trained Models: A Surveycited by 151
  19. 2022Task Residual for Tuning Vision-Language Modelscited by 122
  20. 2023Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimalitycited by 121
  21. 2024Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learningcited by 115
  22. 2023Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Modelscited by 112
  23. 2024ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generationcited by 74
  24. 2022Overcoming Catastrophic Forgetting in Incremental Object Detection via Elastic Response Distillationcited by 72
  25. 2023Continual Detection Transformer for Incremental Object Detectioncited by 59
  26. 2024MagicTime: Time-lapse Video Generation Models as Metamorphic Simulatorscited by 51
  27. 2023Introducing Language Guidance in Prompt-based Continual Learningcited by 49
  28. 2024SDDGR: Stable Diffusion-based Deep Generative Replay for Class Incremental Object Detectioncited by 42
  29. 2023When Prompt-based Incremental Learning Does Not Meet Strong Pretrainingcited by 41
  30. 2023Weighted Ensemble Models Are Strong Continual Learnerscited by 41
  31. 2023Augmented Box Replay: Overcoming Foreground Shift for Incremental Object Detectioncited by 32
  32. 2022Modeling Missing Annotations for Incremental Learning in Object Detectioncited by 23
  33. 2023Learning Hierarchical Prompt with Structured Linguistic Knowledge for Vision-Language Modelscited by 23
  34. 2024Zero-shot Generalizable Incremental Learning for Vision-Language Object Detectioncited by 15
  35. 2023LOVM: Language-Only Vision Model Selectioncited by 12
  36. 2023Open World Object Detection in the Era of Foundation Modelscited by 9
  37. 2023Read Between the Layers: Leveraging Multi-Layer Representations for Rehearsal-Free Continual Learning with Pre-Trained Modelscited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Attributes Should Come from Images, Not Class Names: Distribution-Conditioned Attribute Selection for Vision-Language Models
  2. 2023Investigating the Role of Attribute Context in Vision-Language Models for Object Recognition and Detection
  3. 2025Compositional Caching for Training-free Open-vocabulary Attribute Detection
  4. 2025Roboflow100-VL: A Multi-Domain Object Detection Benchmark for Vision-Language Models
  5. 2026C-GAP: Class-Aware and Online Prompting Improves Vision-Language Models on Imbalanced Classes
  6. 2026ECAD: Expanding Class-Agnostic Detection Beyond Thing-Centric Objectness
  7. 2025Black Sheep in the Herd: Playing with Spuriously Correlated Attributes for Vision-Language Recognition
  8. 2024Real Classification by Description: Extending CLIP's Limits of Part Attributes Recognition