mlsanityData to 7 Oct 2026

No time to train! Training-Free Reference-Based Instance Segmentation

arXiv 2507.02798 ↗First posted 3 Jul 2025cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

The performance of image segmentation models has historically been constrained by the high cost of collecting large-scale annotated data. The Segment Anything Model (SAM) alleviates this original problem through a promptable, semantics-agnostic, segmentation paradigm and yet still requires manual visual-prompts or complex domain-dependent prompt-generation rules to process a new image. Towards reducing this new burden, our work investigates the task of object segmentation when provided with, alternatively, only a small set of reference images. Our key insight is to leverage strong semantic priors, as learned by foundation models, to identify corresponding regions between a reference and a target image. We find that correspondences enable automatic generation of instance-level segmentation masks for downstream tasks and instantiate our ideas via a multi-stage, training-free method incorporating (1) memory bank construction; (2) representation aggregation and (3) semantic-aware feature matching. Our experiments show significant improvements on segmentation metrics, leading to state-of-the-art performance on COCO FSOD (36.8% nAP), PASCAL VOC Few-Shot (71.2% nAP50) and outperforming existing training-free approaches on the Cross-Domain FSOD benchmark (22.4% nAP).

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 80 of its 94 references that are in the map. First posted on arXiv 3 Jul 2025.

Builds on

80 of 94 references.

References · 80

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2023Segment Anythingcited by 8,158
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2017Mask R-CNNcited by 7,259
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  8. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  9. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  10. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  11. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  12. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
38 more
  1. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  2. 2022Visual Prompt Tuningcited by 1,674
  3. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  4. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  5. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
  6. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
  7. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 613
  8. 2023Segment Everything Everywhere All at Oncecited by 546
  9. 2019PANet: Few-Shot Image Semantic Segmentation with Prototype Alignmentcited by 449
  10. 2023Faster Segment Anything: Towards Lightweight SAM for Mobile Applicationscited by 345
  11. 2022Images Speak in Images: A Generalist Painter for In-Context Visual Learningcited by 317
  12. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 315
  13. 2020Frustratingly Simple Few-Shot Object Detectioncited by 280
  14. 2018Few-shot Object Detection via Feature Reweightingcited by 265
  15. 2023Personalize Segment Anything Model with One Shotcited by 250
  16. 2023What does CLIP know about a red circle? Visual prompt engineering for VLMscited by 215
  17. 2019Few-Shot Object Detection with Attention-RPN and Multi-Relation Detectorcited by 206
  18. 2021Hypercorrelation Squeeze for Few-Shot Segmentationcited by 193
  19. 2023SegGPT: Segmenting Everything In Contextcited by 193
  20. 2019Feature Weighting and Boosting for Few-Shot Segmentationcited by 189
  21. 2023Semantic-SAM: Segment and Recognize Anything at Any Granularitycited by 185
  22. 2019Meta R-CNN : Towards General Solver for Instance-level Few-shot Learningcited by 182
  23. 2021CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Modelscited by 182
  24. 2021FSCE: Few-Shot Object Detection via Contrastive Proposal Encodingcited by 168
  25. 2020Few-Shot Object Detection and Viewpoint Estimation for Objects in the Wildcited by 142
  26. 2023Matcher: Segment Anything with One Shot Using All-Purpose Feature Matchingcited by 142
  27. 2021Few-Shot Segmentation via Cycle-Consistent Transformercited by 133
  28. 2016Fully Convolutional Instance-aware Semantic Segmentationcited by 122
  29. 2023Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applicationscited by 121
  30. 2022Learning What Not to Segment: A New Perspective on Few-Shot Segmentationcited by 120
  31. 2022Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentationcited by 114
  32. 2019Large-scale interactive object segmentation with human annotatorscited by 109
  33. 2021DeFRCN: Decoupled Faster R-CNN for Few-Shot Object Detectioncited by 106
  34. 2019TensorMask: A Foundation for Dense Object Segmentationcited by 101
  35. 2024Segment Anything Model for Medical Image Segmentation: Current Applications and Future Directionscited by 97
  36. 2024T-Rex2: Towards Generic Object Detection via Text-Visual Prompt Synergycited by 94
  37. 2022Self-Supervised Learning of Object Parts for Semantic Segmentationcited by 90
  38. 2021Generalized Few-Shot Object Detection without Forgettingcited by 88

50 most cited of 80 All 80 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026INSID3: Training-Free In-Context Segmentation with DINOv3cited by 12
  2. 2026FSOD-VFM: Few-Shot Object Detection with Vision Foundation Models and Graph Diffusioncited by 2
  3. 2026A Training-Free Guess What Vision Language Model from Snippets to Open-Vocabulary Object Detectioncited by 1
  4. 2026SegRAG: Training-Free Retrieval-Augmented Semantic Segmentationcited by 1
  5. 2026ZODS-RS -- Zero-training Oriented Detection & Segmentation for Remote Sensingcited by 0
  6. 2026Boundary-by-Mask: Few-Shot Instance Segmentation with Mask-Conditioned Boundary Learning for Texture-Poor Industrial Partscited by 0
  7. 2026Personalized Object Identification and Localization via In-Context Inference with Vision-Language Modelscited by 0
  8. 2026VocaDet: Sample-Driven Open-Vocabulary Object Detection and Segmentation via Visual Tokenization and Vector Database Retrievalcited by 0
  9. 2026FoRIS: Progressive Foreground Refinement for Training-Free In-Context Segmentationcited by 0