# Explicit Visual Prompting for Low-Level Structure Segmentations

arXiv: https://arxiv.org/abs/2303.10883 · HTML: https://mlsanity.com/paper/2303.10883/

## Key facts

- Cited by 114 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 90 references that are in the map.
- First posted on arXiv 20 Mar 2023.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition](https://mlsanity.com/paper/2205.13535/) (2022)
- [Visual Prompting via Image Inpainting](https://mlsanity.com/paper/2209.00647/) (2022)
- [Anabranch Network for Camouflaged Object Segmentation](https://mlsanity.com/paper/2105.09451/) (2021)
- [Simultaneously Localize, Segment and Rank the Camouflaged Objects](https://mlsanity.com/paper/2103.04011/) (2021)
- [Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal](https://mlsanity.com/paper/1712.02478/) (2017)
- [Learning Rich Features for Image Manipulation Detection](https://mlsanity.com/paper/1805.04953/) (2018)
- [Camouflaged Object Segmentation with Distraction Mining](https://mlsanity.com/paper/2104.10475/) (2021)
- [Fighting Fake News: Image Splice Detection via Learned Self-Consistency](https://mlsanity.com/paper/1805.04096/) (2018)
- [PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2103.10596/) (2021)
- [Uncertainty-aware Joint Salient Object and Camouflaged Object Detection](https://mlsanity.com/paper/2104.02628/) (2021)
- [Towards Ghost-free Shadow Removal via Dual Hierarchical Aggregation Network and Shadow Matting GAN](https://mlsanity.com/paper/1911.08718/) (2019)
- [SPAN: Spatial Pyramid Attention Network forImage Manipulation Localization](https://mlsanity.com/paper/2009.00726/) (2020)
- [ObjectFormer for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2203.14681/) (2022)
- [Image Splicing Localization Using A Multi-Task Fully Convolutional Network (MFCN)](https://mlsanity.com/paper/1709.02016/) (2017)
- [Direction-aware Spatial Context Features for Shadow Detection](https://mlsanity.com/paper/1712.04142/) (2017)
- [A+D Net: Training a Shadow Detector with Adversarial Shadow Attenuation](https://mlsanity.com/paper/1712.01361/) (2017)
- [Fine-tuning Image Transformers using Learnable Memory](https://mlsanity.com/paper/2203.15243/) (2022)

## Built on by (most cited first, 30 of 114)

- [Perceive-IR: Learning to Perceive Degradation Better for All-in-One Image Restoration](https://mlsanity.com/paper/2408.15994/) (2024)
- [SAM2-Adapter: Evaluating & Adapting Segment Anything 2 in Downstream Tasks: Camouflage, Shadow, Medical Image Segmentation, and More](https://mlsanity.com/paper/2408.04579/) (2024)
- [A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trends](https://mlsanity.com/paper/2410.15067/) (2024)
- [DiffMOT: A Real-time Diffusion-based Multiple Object Tracker with Non-linear Prediction](https://mlsanity.com/paper/2403.02075/) (2024)
- [Visual Prompting in Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2409.15310/) (2024)
- [SAM Fails to Segment Anything? -- SAM-Adapter: Adapting SAM in Underperformed Scenes: Camouflage, Shadow, Medical Image Segmentation, and More](https://mlsanity.com/paper/2304.09148/) (2023)
- [Annotation-free Audio-Visual Segmentation](https://mlsanity.com/paper/2305.11019/) (2023)
- [Frequency-Spatial Entanglement Learning for Camouflaged Object Detection](https://mlsanity.com/paper/2409.01686/) (2024)
- [Parameter-Efficient Fine-Tuning for Foundation Models](https://mlsanity.com/paper/2501.13787/) (2025)
- [Seg-R1: Segmentation Can Be Surprisingly Simple with Reinforcement Learning](https://mlsanity.com/paper/2506.22624/) (2025)
- [All-in-SAM: from Weak Annotation to Pixel-wise Nuclei Segmentation with Prompt-based Finetuning](https://mlsanity.com/paper/2307.00290/) (2023)
- [Pixel-Inconsistency Modeling for Image Manipulation Localization](https://mlsanity.com/paper/2310.00234/) (2023)
- [CAT-SAM: Conditional Tuning for Few-Shot Adaptation of Segment Anything Model](https://mlsanity.com/paper/2402.03631/) (2024)
- [Tuning Multi-mode Token-level Prompt Alignment across Modalities](https://mlsanity.com/paper/2309.13847/) (2023)
- [CoLLaVO: Crayon Large Language and Vision mOdel](https://mlsanity.com/paper/2402.11248/) (2024)
- [Depth Prompting for Sensor-Agnostic Depth Estimation](https://mlsanity.com/paper/2405.11867/) (2024)
- [Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective](https://mlsanity.com/paper/2312.01397/) (2023)
- [Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models: A Survey and Benchmark](https://mlsanity.com/paper/2402.02242/) (2024)
- [A Simple yet Effective Network based on Vision Transformer for Camouflaged Object and Salient Object Detection](https://mlsanity.com/paper/2402.18922/) (2024)
- [Prompt-based Adaptation in Large-scale Vision Models: A Survey](https://mlsanity.com/paper/2510.13219/) (2025)
- [Self-Sampling Meta SAM: Enhancing Few-shot Medical Image Segmentation with Meta-Learning](https://mlsanity.com/paper/2308.16466/) (2023)
- [Spider: A Unified Framework for Context-dependent Concept Segmentation](https://mlsanity.com/paper/2405.01002/) (2024)
- [Explicit Visual Prompting for Universal Foreground Segmentations](https://mlsanity.com/paper/2305.18476/) (2023)
- [CamDiff: Camouflage Image Augmentation via Diffusion Model](https://mlsanity.com/paper/2304.05469/) (2023)
- [FisherTune: Fisher-Guided Robust Tuning of Vision Foundation Models for Domain Generalized Segmentation](https://mlsanity.com/paper/2503.17940/) (2025)
- [Exploring Multi-view Pixel Contrast for General and Robust Image Forgery Localization](https://mlsanity.com/paper/2406.13565/) (2024)
- [Underwater Image Restoration via Polymorphic Large Kernel CNNs](https://mlsanity.com/paper/2412.18459/) (2024)
- [Improving SAM for Camouflaged Object Detection via Dual Stream Adapters](https://mlsanity.com/paper/2503.06042/) (2025)
- [High-Fidelity Document Stain Removal via A Large-Scale Real-World Dataset and A Memory-Augmented Transformer](https://mlsanity.com/paper/2410.22922/) (2024)
- [DVPT: Dynamic Visual Prompt Tuning of Large Pre-trained Models for Medical Image Analysis](https://mlsanity.com/paper/2307.09787/) (2023)

## Abstract (arXiv)

We consider the generic problem of detecting low-level structures in images, which includes segmenting the manipulated parts, identifying out-of-focus pixels, separating shadow regions, and detecting concealed objects. Whereas each such topic has been typically addressed with a domain-specific solution, we show that a unified approach performs well across all of them. We take inspiration from the widely-used pre-training and then prompt tuning protocols in NLP and propose a new visual prompting model, named Explicit Visual Prompting (EVP). Different from the previous visual prompting which is typically a dataset-level implicit embedding, our key insight is to enforce the tunable parameters focusing on the explicit visual content from each individual image, i.e., the features from frozen patch embeddings and the input's high-frequency components. The proposed EVP significantly outperforms other parameter-efficient tuning protocols under the same amount of tunable parameters (5.7% extra trainable parameters of each task). EVP also achieves state-of-the-art performances on diverse low-level structure segmentation tasks compared to task-specific solutions. Our code is available at: https://github.com/NiFangBaAGe/Explicit-Visual-Prompt.