# Unifying Image Processing as Visual Prompting Question Answering

arXiv: https://arxiv.org/abs/2310.10513 · HTML: https://mlsanity.com/paper/2310.10513/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 50 references that are in the map.
- First posted on arXiv 16 Oct 2023.

## Builds on (most cited first, 30 of 35)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Deep Retinex Decomposition for Low-Light Enhancement](https://mlsanity.com/paper/1808.04560/) (2018)
- [Designing a Practical Degradation Model for Deep Blind Image Super-Resolution](https://mlsanity.com/paper/2103.14006/) (2021)
- [InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions](https://mlsanity.com/paper/2211.05778/) (2022)
- [Benchmarking Single Image Dehazing and Beyond](https://mlsanity.com/paper/1712.04143/) (2017)
- [InternVideo: General Video Foundation Models via Generative and Discriminative Learning](https://mlsanity.com/paper/2212.03191/) (2022)
- [Accelerating the Super-Resolution Convolutional Neural Network](https://mlsanity.com/paper/1608.00367/) (2016)
- [Larger language models do in-context learning differently](https://mlsanity.com/paper/2303.03846/) (2023)
- [DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks](https://mlsanity.com/paper/1711.07064/) (2017)
- [Sketch-based Manga Retrieval using Manga109 Dataset](https://mlsanity.com/paper/1510.04389/) (2015)
- [Images Speak in Images: A Generalist Painter for In-Context Visual Learning](https://mlsanity.com/paper/2212.02499/) (2022)
- [Visual Prompting via Image Inpainting](https://mlsanity.com/paper/2209.00647/) (2022)
- [Track Anything: Segment Anything Meets Videos](https://mlsanity.com/paper/2304.11968/) (2023)
- [Inpaint Anything: Segment Anything Meets Image Inpainting](https://mlsanity.com/paper/2304.06790/) (2023)
- [Unsupervised Degradation Representation Learning for Blind Super-Resolution](https://mlsanity.com/paper/2104.00416/) (2021)
- [What Makes Good Examples for Visual In-Context Learning?](https://mlsanity.com/paper/2301.13670/) (2023)

## Built on by (most cited first, 29 of 29)

- [A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trends](https://mlsanity.com/paper/2410.15067/) (2024)
- [Towards Effective Multiple-in-One Image Restoration: A Sequential and Prompt Learning Strategy](https://mlsanity.com/paper/2401.03379/) (2024)
- [An Intelligent Agentic System for Complex Image Restoration Problems](https://mlsanity.com/paper/2410.17809/) (2024)
- [VisualCloze: A Universal Image Generation Framework via Visual In-Context Learning](https://mlsanity.com/paper/2504.07960/) (2025)
- [DocRes: A Generalist Model Toward Unifying Document Image Restoration Tasks](https://mlsanity.com/paper/2405.04408/) (2024)
- [Learning A Low-Level Vision Generalist via Visual Task Prompt](https://mlsanity.com/paper/2408.08601/) (2024)
- [LoRA-IR: Taming Low-Rank Experts for Efficient All-in-One Image Restoration](https://mlsanity.com/paper/2410.15385/) (2024)
- [Learning Continuous Wasserstein Barycenter Space for Generalized All-in-One Image Restoration](https://mlsanity.com/paper/2602.23169/) (2026)
- [Lumina-OmniLV: A Unified Multimodal Framework for General Low-Level Vision](https://mlsanity.com/paper/2504.04903/) (2025)
- [GRIDS: Grouped Multiple-Degradation Restoration with Image Degradation Similarity](https://mlsanity.com/paper/2407.12273/) (2024)
- [WeatherGFM: Learning A Weather Generalist Foundation Model via In-context Learning](https://mlsanity.com/paper/2411.05420/) (2024)
- [PairEdit: Learning Semantic Variations for Exemplar-based Image Editing](https://mlsanity.com/paper/2506.07992/) (2025)
- [Parameter Efficient Adaptation for Image Restoration with Heterogeneous Mixture-of-Experts](https://mlsanity.com/paper/2312.08881/) (2023)
- [A Preliminary Exploration Towards General Image Restoration](https://mlsanity.com/paper/2408.15143/) (2024)
- [LoRA of Change: Learning to Generate LoRA for the Editing Instruction from A Single Before-After Image Pair](https://mlsanity.com/paper/2411.19156/) (2024)
- [Uni-DocDiff: A Unified Document Restoration Model Based on Diffusion](https://mlsanity.com/paper/2508.04055/) (2025)
- [Spanning the Visual Analogy Space with a Weight Basis of LoRAs](https://mlsanity.com/paper/2602.15727/) (2026)
- [PromptSR: Cascade Prompting for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2507.04118/) (2025)
- [Exploring Scalable Unified Modeling for General Low-Level Vision](https://mlsanity.com/paper/2507.14801/) (2025)
- [ReviveDiff: A Universal Diffusion Model for Restoring Images in Adverse Weather Conditions](https://mlsanity.com/paper/2409.18932/) (2024)
- [BaryIR: Learning Multi-Source Unified Representation in Continuous Barycenter Space for Generalizable All-in-One Image Restoration](https://mlsanity.com/paper/2505.21637/) (2025)
- [ConStyle v2: A Strong Prompter for All-in-One Image Restoration](https://mlsanity.com/paper/2406.18242/) (2024)
- [Deep Lidar-guided Image Deblurring](https://mlsanity.com/paper/2412.07262/) (2024)
- [EditTransfer++: Toward Faithful and Efficient Visual-Prompt-Guided Image Editing](https://mlsanity.com/paper/2605.07455/) (2026)
- [A Simple Framework Uniting Visual In-context Learning with Masked Image Modeling to Improve Ultrasound Segmentation](https://mlsanity.com/paper/2402.14300/) (2024)
- [Learning Dual Transformers for All-In-One Image Restoration from a Frequency Perspective](https://mlsanity.com/paper/2407.01636/) (2024)
- [FlexICL: A Flexible Visual In-context Learning Framework for Elbow and Wrist Ultrasound Segmentation](https://mlsanity.com/paper/2510.26049/) (2025)
- [Unleashing Degradation-Carrying Features in Symmetric U-Net: Simpler and Stronger Baselines for All-in-One Image Restoration](https://mlsanity.com/paper/2512.10581/) (2025)
- [Hidden-Shot: Towards One-Shot Task Generalization for Low-Level Vision Generalist Models](https://mlsanity.com/paper/2607.01535/) (2026)

## Abstract (arXiv)

Image processing is a fundamental task in computer vision, which aims at enhancing image quality and extracting essential features for subsequent vision applications. Traditionally, task-specific models are developed for individual tasks and designing such models requires distinct expertise. Building upon the success of large language models (LLMs) in natural language processing (NLP), there is a similar trend in computer vision, which focuses on developing large-scale models through pretraining and in-context learning. This paradigm shift reduces the reliance on task-specific models, yielding a powerful unified model to deal with various tasks. However, these advances have predominantly concentrated on high-level vision tasks, with less attention paid to low-level vision tasks. To address this issue, we propose a universal model for general image processing that covers image restoration, image enhancement, image feature extraction tasks, etc. Our proposed framework, named PromptGIP, unifies these diverse image processing tasks within a universal framework. Inspired by NLP question answering (QA) techniques, we employ a visual prompting question answering paradigm. Specifically, we treat the input-output image pair as a structured question-answer sentence, thereby reprogramming the image processing task as a prompting QA problem. PromptGIP can undertake diverse cross-domain tasks using provided visual prompts, eliminating the need for task-specific finetuning. Our methodology offers a universal and adaptive solution to general image processing. While PromptGIP has demonstrated a certain degree of out-of-domain task generalization capability, further research is expected to fully explore its more powerful emergent generalization.