# Spider: A Unified Framework for Context-dependent Concept Segmentation

arXiv: https://arxiv.org/abs/2405.01002 · HTML: https://mlsanity.com/paper/2405.01002/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 77 of its 119 references that are in the map.
- First posted on arXiv 2 May 2024.

## Builds on (most cited first, 30 of 77)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [PVT v2: Improved Baselines with Pyramid Vision Transformer](https://mlsanity.com/paper/2106.13797/) (2021)
- [Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1902.03368/) (2019)
- [Image Segmentation Using Text and Image Prompts](https://mlsanity.com/paper/2112.10003/) (2021)
- [InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions](https://mlsanity.com/paper/2211.05778/) (2022)
- [Structure-measure: A New Way to Evaluate Foreground Maps](https://mlsanity.com/paper/1708.00786/) (2017)
- [Res2Net: A New Multi-scale Backbone Architecture](https://mlsanity.com/paper/1904.01169/) (2019)
- [PraNet: Parallel Reverse Attention Network for Polyp Segmentation](https://mlsanity.com/paper/2006.11392/) (2020)
- [Images Speak in Images: A Generalist Painter for In-Context Visual Learning](https://mlsanity.com/paper/2212.02499/) (2022)
- [CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection](https://mlsanity.com/paper/2301.00785/) (2023)
- [Multi-scale Interactive Network for Salient Object Detection](https://mlsanity.com/paper/2007.09062/) (2020)
- [Simultaneously Localize, Segment and Rank the Camouflaged Objects](https://mlsanity.com/paper/2103.04011/) (2021)
- [Universal Instance Perception as Object Discovery and Retrieval](https://mlsanity.com/paper/2303.06674/) (2023)
- [Inf-Net: Automatic COVID-19 Lung Infection Segmentation from CT Images](https://mlsanity.com/paper/2004.14133/) (2020)

## Built on by (most cited first, 18 of 18)

- [Controllable-LPMoE: Adapting to Challenging Object Segmentation via Dynamic Local Priors from Mixture-of-Experts](https://mlsanity.com/paper/2510.21114/) (2025)
- [Rethinking Detecting Salient and Camouflaged Objects in Unconstrained Scenes](https://mlsanity.com/paper/2412.10943/) (2024)
- [TP-Seg: Task-Prototype Framework for Unified Medical Lesion Segmentation](https://mlsanity.com/paper/2604.00684/) (2026)
- [Deep Learning in Concealed Dense Prediction](https://mlsanity.com/paper/2504.10979/) (2025)
- [DifferSeg: Towards Diverse Multimodal Binary Segmentation via Differential Perception and Frequency Guidance](https://mlsanity.com/paper/2606.08906/) (2026)
- [Beyond Single Images: Retrieval Self-Augmented Unsupervised Camouflaged Object Detection](https://mlsanity.com/paper/2510.18437/) (2025)
- [Refining Context-Entangled Content Segmentation via Curriculum Selection and Anti-Curriculum Promotion](https://mlsanity.com/paper/2602.01183/) (2026)
- [Segment Concealed Objects with Incomplete Supervision](https://mlsanity.com/paper/2506.08955/) (2025)
- [UniMMAD: Unified Multi-Modal and Multi-Class Anomaly Detection via MoE-Driven Feature Decompression](https://mlsanity.com/paper/2509.25934/) (2025)
- [Marine Saliency Segmenter: Object-Focused Conditional Diffusion with Region-Level Semantic Knowledge Distillation](https://mlsanity.com/paper/2504.02391/) (2025)
- [Beyond Pixel Overlap: A Framework for Decomposing Segmentation Evaluation Metrics](https://mlsanity.com/paper/2607.00886/) (2026)
- [Fourier Boundary Features Network with Wider Catchers for Glass Segmentation](https://mlsanity.com/paper/2405.09459/) (2024)
- [Concept-to-Pixel: Prompt-Free Universal Medical Image Segmentation](https://mlsanity.com/paper/2603.17746/) (2026)
- [RIDE: Retinex-Informed Decoupling for Exposing Concealed Objects](https://mlsanity.com/paper/2605.15450/) (2026)
- [Hierarchical Consistency Learning for Test-time Adaptation in Camouflage Perception](https://mlsanity.com/paper/2605.25651/) (2026)
- [Explainable Forensics of Manipulated Segments in Untrimmed Long Videos](https://mlsanity.com/paper/2606.02402/) (2026)
- [Boundary-by-Mask: Few-Shot Instance Segmentation with Mask-Conditioned Boundary Learning for Texture-Poor Industrial Parts](https://mlsanity.com/paper/2606.21594/) (2026)
- [Is There Really a Camouflaged Object? Towards Realistic Camouflaged Object Detection](https://mlsanity.com/paper/2608.11135/) (2026)

## Abstract (arXiv)

Different from the context-independent (CI) concepts such as human, car, and airplane, context-dependent (CD) concepts require higher visual understanding ability, such as camouflaged object and medical lesion. Despite the rapid advance of many CD understanding tasks in respective branches, the isolated evolution leads to their limited cross-domain generalisation and repetitive technique innovation. Since there is a strong coupling relationship between foreground and background context in CD tasks, existing methods require to train separate models in their focused domains. This restricts their real-world CD concept understanding towards artificial general intelligence (AGI). We propose a unified model with a single set of parameters, Spider, which only needs to be trained once. With the help of the proposed concept filter driven by the image-mask group prompt, Spider is able to understand and distinguish diverse strong context-dependent concepts to accurately capture the Prompter's intention. Without bells and whistles, Spider significantly outperforms the state-of-the-art specialized models in 8 different context-dependent segmentation tasks, including 4 natural scenes (salient, camouflaged, and transparent objects and shadow) and 4 medical lesions (COVID-19, polyp, breast, and skin lesion with color colonoscopy, CT, ultrasound, and dermoscopy modalities). Besides, Spider shows obvious advantages in continuous learning. It can easily complete the training of new tasks by fine-tuning parameters less than 1% and bring a tolerable performance degradation of less than 5% for all old tasks. The source code will be publicly available at hrefhttps://github.com/Xiaoqi-Zhao-DLUT/Spider-UniCDSegSpider-UniCDSeg.