mlsanityData to 14 Oct 2026

Re-Scoring Using Image-Language Similarity for Few-Shot Object Detection

arXiv 2311.00278 ↗First posted 1 Nov 2023cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Few-shot object detection, which focuses on detecting novel objects with few labels, is an emerging challenge in the community. Recent studies show that adapting a pre-trained model or modified loss function can improve performance. In this paper, we explore leveraging the power of Contrastive Language-Image Pre-training (CLIP) and hard negative classification loss in low data setting. Specifically, we propose Re-scoring using Image-language Similarity for Few-shot object detection (RISF) which extends Faster R-CNN by introducing Calibration Module using CLIP (CM-CLIP) and Background Negative Re-scale Loss (BNRL). The former adapts CLIP, which performs zero-shot classification, to re-score the classification scores of a detector using image-class similarities, the latter is modified classification loss considering the punishment for fake backgrounds as well as confusing categories on a generalized few-shot object detection dataset. Extensive experiments on MS-COCO and PASCAL VOC show that the proposed RISF substantially outperforms the state-of-the-art approaches. The code will be available.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 49 of its 61 references that are in the map. First posted on arXiv 1 Nov 2023.

Builds on

49 of 61 references.

References · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  8. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  9. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  10. 2020End-to-End Object Detection with Transformerscited by 7,414
  11. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  12. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
37 more
  1. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  2. 2015Fast R-CNNcited by 2,933
  3. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  4. 2016Matching Networks for One Shot Learningcited by 2,255
  5. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  6. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  7. 2016YOLO9000: Better, Faster, Strongercited by 1,431
  8. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,276
  9. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
  10. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 906
  11. 2019A Closer Look at Few-shot Classificationcited by 684
  12. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 616
  13. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 602
  14. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
  15. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 437
  16. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  17. 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
  18. 2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 379
  19. 2020Frustratingly Simple Few-Shot Object Detectioncited by 282
  20. 2018Few-shot Object Detection via Feature Reweightingcited by 265
  21. 2019Visual Semantic Reasoning for Image-Text Matchingcited by 185
  22. 2021FSCE: Few-Shot Object Detection via Contrastive Proposal Encodingcited by 168
  23. 2020Few-Shot Object Detection and Viewpoint Estimation for Objects in the Wildcited by 142
  24. 2020Multi-Scale Positive Sample Refinement for Few-Shot Object Detectioncited by 116
  25. 2021DeFRCN: Decoupled Faster R-CNN for Few-Shot Object Detectioncited by 106
  26. 2022Few-Shot Object Detection with Fully Cross-Transformercited by 64
  27. 2021Query Adaptive Few-Shot Object Detection with Heterogeneous Graph Convolutional Networkscited by 55
  28. 2021Meta-DETR: Image-Level Few-Shot Object Detection with Inter-Class Correlation Exploitationcited by 44
  29. 2021Label, Verify, Correct: A Simple Few Shot Object Detection Methodcited by 43
  30. 2021Few-Shot Object Detection via Association and DIscriminationcited by 39
  31. 2021Dual-Awareness Attention for Few-Shot Object Detectioncited by 29
  32. 2020Solving Missing-Annotation Object Detection with Background Recalibration Losscited by 27
  33. 2020Localizing the Common Action Among a Few Videoscited by 11
  34. 2022Multi-Faceted Distillation of Base-Novel Commonality for Few-shot Object Detectioncited by 11
  35. 2022Time-rEversed diffusioN tEnsor Transformer: A new TENET of Few-Shot Object Detectioncited by 10
  36. 2020MM-FSOD: Meta and metric integrated few-shot object detectioncited by 4
  37. 2023An Effective Crop-Paste Pipeline for Few-shot Object Detectioncited by 3

Built on by

Not cited yet.