# A Simple Image Segmentation Framework via In-Context Examples

arXiv: https://arxiv.org/abs/2410.04842 · HTML: https://mlsanity.com/paper/2410.04842/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 59 of its 69 references that are in the map.
- First posted on arXiv 7 Oct 2024.

## Builds on (most cited first, 30 of 59)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [Panoptic Feature Pyramid Networks](https://mlsanity.com/paper/1901.02446/) (2019)
- [Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation](https://mlsanity.com/paper/2012.07177/) (2020)
- [Segment Everything Everywhere All at Once](https://mlsanity.com/paper/2304.06718/) (2023)
- [Scaling Open-Vocabulary Image Segmentation with Image-Level Labels](https://mlsanity.com/paper/2112.12143/) (2021)
- [RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation](https://mlsanity.com/paper/1611.06612/) (2016)
- [YOLACT: Real-time Instance Segmentation](https://mlsanity.com/paper/1904.02689/) (2019)
- [XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Model](https://mlsanity.com/paper/2207.07115/) (2022)
- [YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark](https://mlsanity.com/paper/1809.03327/) (2018)
- [One-Shot Learning for Semantic Segmentation](https://mlsanity.com/paper/1709.03410/) (2017)

## Built on by (most cited first, 19 of 19)

- [SegAgent: Exploring Pixel Understanding Capabilities in MLLMs by Imitating Human Annotator Trajectories](https://mlsanity.com/paper/2503.08625/) (2025)
- [VideoGen-Eval: Agent-based System for Video Generation Evaluation](https://mlsanity.com/paper/2503.23452/) (2025)
- [INSID3: Training-Free In-Context Segmentation with DINOv3](https://mlsanity.com/paper/2603.28480/) (2026)
- [Unlocking the Power of SAM 2 for Few-Shot Segmentation](https://mlsanity.com/paper/2505.14100/) (2025)
- [Training-free Detection of AI-generated images via Cropping Robustness](https://mlsanity.com/paper/2511.14030/) (2025)
- [No time to train! Training-Free Reference-Based Instance Segmentation](https://mlsanity.com/paper/2507.02798/) (2025)
- [SANSA: Unleashing the Hidden Semantics in SAM2 for Few-Shot Segmentation](https://mlsanity.com/paper/2505.21795/) (2025)
- [There is no SAMantics! Exploring SAM as a Backbone for Visual Understanding Tasks](https://mlsanity.com/paper/2411.15288/) (2024)
- [Unified Open-World Segmentation with Multi-Modal Prompts](https://mlsanity.com/paper/2510.10524/) (2025)
- [Show or Tell? A Benchmark To Evaluate Visual and Textual Prompts in Semantic Segmentation](https://mlsanity.com/paper/2505.06280/) (2025)
- [R2Seg: Training-Free OOD Medical Tumor Segmentation via Anatomical Reasoning and Statistical Rejection](https://mlsanity.com/paper/2511.12691/) (2025)
- [Unify the Views: View-Consistent Prototype Learning for Few-Shot Segmentation](https://mlsanity.com/paper/2603.05952/) (2026)
- [UniSpector: Towards Universal Open-set Defect Recognition via Spectral-Contrastive Visual Prompting](https://mlsanity.com/paper/2604.02905/) (2026)
- [Reference-based Category Discovery: Unsupervised Object Detection with Category Awareness](https://mlsanity.com/paper/2605.04606/) (2026)
- [PrAda: Few-Shot Visual Adaptation for Text-Prompted Segmentation](https://mlsanity.com/paper/2605.19623/) (2026)
- [Toward Robust In-Context Segmentation via Concept Guidance](https://mlsanity.com/paper/2606.28149/) (2026)
- [FROST: Training-Free Few-Shot Segmentation with Frozen Features and Nonparametric Statistics](https://mlsanity.com/paper/2606.31136/) (2026)
- [REBASE: Reference-Background Subspace Elimination for Training-Free In-Context Segmentation](https://mlsanity.com/paper/2607.09082/) (2026)
- [FoRIS: Progressive Foreground Refinement for Training-Free In-Context Segmentation](https://mlsanity.com/paper/2609.03384/) (2026)

## Abstract (arXiv)

Recently, there have been explorations of generalist segmentation models that can effectively tackle a variety of image segmentation tasks within a unified in-context learning framework. However, these methods still struggle with task ambiguity in in-context segmentation, as not all in-context examples can accurately convey the task information. In order to address this issue, we present SINE, a simple image Segmentation framework utilizing in-context examples. Our approach leverages a Transformer encoder-decoder structure, where the encoder provides high-quality image representations, and the decoder is designed to yield multiple task-specific output masks to effectively eliminate task ambiguity. Specifically, we introduce an In-context Interaction module to complement in-context information and produce correlations between the target image and the in-context example and a Matching Transformer that uses fixed matching and a Hungarian algorithm to eliminate differences between different tasks. In addition, we have further perfected the current evaluation system for in-context image segmentation, aiming to facilitate a holistic appraisal of these models. Experiments on various segmentation tasks show the effectiveness of the proposed method.