# ReVision: A Dataset and Baseline VLM for Privacy-Preserving Task-Oriented Visual Instruction Rewriting

arXiv: https://arxiv.org/abs/2502.14780 · HTML: https://mlsanity.com/paper/2502.14780/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 38 references that are in the map.
- First posted on arXiv 20 Feb 2025.

## Builds on (most cited first, 23 of 23)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [PaliGemma: A versatile 3B VLM for transfer](https://mlsanity.com/paper/2407.07726/) (2024)
- [Character Region Awareness for Text Detection](https://mlsanity.com/paper/1904.01941/) (2019)
- [PaLI-X: On Scaling up a Multilingual Vision and Language Model](https://mlsanity.com/paper/2305.18565/) (2023)
- [PaliGemma 2: A Family of Versatile VLMs for Transfer](https://mlsanity.com/paper/2412.03555/) (2024)
- [A Survey on Semantic Parsing](https://mlsanity.com/paper/1812.00978/) (2018)
- [Relevance of Unsupervised Metrics in Task-Oriented Dialogue for Evaluating Natural Language Generation](https://mlsanity.com/paper/1706.09799/) (2017)
- [ConvGQR: Generative Query Reformulation for Conversational Search](https://mlsanity.com/paper/2305.15645/) (2023)
- [Recent Neural Methods on Slot Filling and Intent Classification for Task-Oriented Dialogue Systems: A Survey](https://mlsanity.com/paper/2011.00564/) (2020)
- [Conversational Semantic Parsing](https://mlsanity.com/paper/2009.13655/) (2020)
- [PRESTO: A Multilingual Dataset for Parsing Realistic Task-Oriented Dialogs](https://mlsanity.com/paper/2303.08954/) (2023)
- [LLM-R2: A Large Language Model Enhanced Rule-based Rewrite System for Boosting Query Efficiency](https://mlsanity.com/paper/2404.12872/) (2024)
- [Benchmarking Vision-Language Models on Optical Character Recognition in Dynamic Video Environments](https://mlsanity.com/paper/2502.06445/) (2025)
- [Generating Natural Questions from Images for Multimodal Assistants](https://mlsanity.com/paper/2012.03678/) (2020)
- [Can Visual Language Models Replace OCR-Based Visual Question Answering Pipelines in Production? A Case Study in Retail](https://mlsanity.com/paper/2408.15626/) (2024)
- [Visual Question Rewriting for Increasing Response Rate](https://mlsanity.com/paper/2106.02257/) (2021)
- [Can Open Domain Question Answering Systems Answer Visual Knowledge Questions?](https://mlsanity.com/paper/2202.04306/) (2022)

## Built on by (most cited first, 6 of 6)

- [Object Detection with Multimodal Large Vision-Language Models: An In-depth Review](https://mlsanity.com/paper/2508.19294/) (2025)
- [When Privacy Meets Recovery: The Overlooked Half of Surrogate-Driven Privacy Preservation for MLLM Editing](https://mlsanity.com/paper/2512.07166/) (2025)
- [Position: Embodied AI Requires a Privacy-Utility Trade-off](https://mlsanity.com/paper/2605.05017/) (2026)
- [See No Evil: Semantic Context-Aware Privacy Risk Detection for AR](https://mlsanity.com/paper/2604.22805/) (2026)
- [When Recovery Matters: The Blind Spot of Surrogate Privacy in MLLM Editing](https://mlsanity.com/paper/2606.07171/) (2026)
- [Seeing Without Exposing: Adaptive Privacy Control for Open-World, Context-Hungry MLLMs](https://mlsanity.com/paper/2606.07175/) (2026)

## Abstract (arXiv)

Efficient and privacy-preserving multimodal interaction is essential as AR, VR, and modern smartphones with powerful cameras become primary interfaces for human-computer communication. Existing powerful large vision-language models (VLMs) enabling multimodal interaction often rely on cloud-based processing, raising significant concerns about (1) visual privacy by transmitting sensitive vision data to servers, and (2) their limited real-time, on-device usability. This paper explores Visual Instruction Rewriting, a novel approach that transforms multimodal instructions into text-only commands, allowing seamless integration of lightweight on-device instruction rewriter VLMs (250M parameters) with existing conversational AI systems, enhancing vision data privacy. To achieve this, we present a dataset of over 39,000 examples across 14 domains and develop a compact VLM, pretrained on image captioning datasets and fine-tuned for instruction rewriting. Experimental results, evaluated through NLG metrics such as BLEU, METEOR, and ROUGE, along with semantic parsing analysis, demonstrate that even a quantized version of the model (<500MB storage footprint) can achieve effective instruction rewriting, thus enabling privacy-focused, multimodal AI applications.