# Multi-granularity Interaction Simulation for Unsupervised Interactive Segmentation

arXiv: https://arxiv.org/abs/2303.13399 · HTML: https://mlsanity.com/paper/2303.13399/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 52 references that are in the map.
- First posted on arXiv 23 Mar 2023.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [Exploring Plain Vision Transformer Backbones for Object Detection](https://mlsanity.com/paper/2203.16527/) (2022)
- [The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification](https://mlsanity.com/paper/2107.02314/) (2021)
- [Dense Contrastive Learning for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2011.09157/) (2020)
- [SOLO: Segmenting Objects by Locations](https://mlsanity.com/paper/1912.04488/) (2019)
- [Unsupervised Semantic Segmentation by Distilling Feature Correspondences](https://mlsanity.com/paper/2203.08414/) (2022)
- [Localizing Objects with Self-Supervised Transformers and no Labels](https://mlsanity.com/paper/2109.14279/) (2021)
- [Self-Supervised Transformers for Unsupervised Object Discovery using Normalized Cut](https://mlsanity.com/paper/2202.11539/) (2022)
- [Reviving Iterative Training with Mask Guidance for Interactive Segmentation](https://mlsanity.com/paper/2102.06583/) (2021)
- [Deep Interactive Object Selection](https://mlsanity.com/paper/1603.04042/) (2016)
- [Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localization](https://mlsanity.com/paper/2205.07839/) (2022)
- [f-BRS: Rethinking Backpropagating Refinement for Interactive Segmentation](https://mlsanity.com/paper/2001.10331/) (2020)
- [SimpleClick: Interactive Image Segmentation with Simple Vision Transformers](https://mlsanity.com/paper/2210.11006/) (2022)
- [FreeSOLO: Learning to Segment Objects without Annotations](https://mlsanity.com/paper/2202.12181/) (2022)
- [FocalClick: Towards Practical Interactive Image Segmentation](https://mlsanity.com/paper/2204.02574/) (2022)
- [The Fast Bilateral Solver](https://mlsanity.com/paper/1511.03296/) (2015)
- [PseudoClick: Interactive Image Segmentation with Click Imitation](https://mlsanity.com/paper/2207.05282/) (2022)
- [EdgeFlow: Achieving Practical Interactive Segmentation with Edge-Guided Flow](https://mlsanity.com/paper/2109.09406/) (2021)
- [Continuous Adaptation for Interactive Object Segmentation by Learning from Corrections](https://mlsanity.com/paper/1911.12709/) (2019)
- [Interactive Segmentation as Gaussian Process Classification](https://mlsanity.com/paper/2302.14578/) (2023)

## Built on by (most cited first, 9 of 9)

- [GraCo: Granularity-Controllable Interactive Segmentation](https://mlsanity.com/paper/2405.00587/) (2024)
- [Repurposing Stable Diffusion Attention for Training-Free Unsupervised Interactive Segmentation](https://mlsanity.com/paper/2411.10411/) (2024)
- [PaintSeg: Training-free Segmentation via Painting](https://mlsanity.com/paper/2305.19406/) (2023)
- [TG-VQA: Ternary Game of Video Question Answering](https://mlsanity.com/paper/2305.10049/) (2023)
- [Upsampling DINOv2 features for unsupervised vision tasks and weakly supervised materials segmentation](https://mlsanity.com/paper/2410.19836/) (2024)
- [PerSense: Training-Free Personalized Instance Segmentation in Dense Images](https://mlsanity.com/paper/2405.13518/) (2024)
- [Towards PerSense++: Advancing Training-Free Personalized Instance Segmentation in Dense Images](https://mlsanity.com/paper/2508.14660/) (2025)
- [M2N2V2: Multi-Modal Unsupervised and Training-free Interactive Segmentation](https://mlsanity.com/paper/2503.16254/) (2025)
- [RefCut: Interactive Segmentation with Reference Guidance](https://mlsanity.com/paper/2503.17820/) (2025)

## Abstract (arXiv)

Interactive segmentation enables users to segment as needed by providing cues of objects, which introduces human-computer interaction for many fields, such as image editing and medical image analysis. Typically, massive and expansive pixel-level annotations are spent to train deep models by object-oriented interactions with manually labeled object masks. In this work, we reveal that informative interactions can be made by simulation with semantic-consistent yet diverse region exploration in an unsupervised paradigm. Concretely, we introduce a Multi-granularity Interaction Simulation (MIS) approach to open up a promising direction for unsupervised interactive segmentation. Drawing on the high-quality dense features produced by recent self-supervised models, we propose to gradually merge patches or regions with similar features to form more extensive regions and thus, every merged region serves as a semantic-meaningful multi-granularity proposal. By randomly sampling these proposals and simulating possible interactions based on them, we provide meaningful interaction at multiple granularities to teach the model to understand interactions. Our MIS significantly outperforms non-deep learning unsupervised methods and is even comparable with some previous deep-supervised methods without any annotation.