# Breaking the Box: Enhancing Remote Sensing Image Segmentation with Freehand Sketches

arXiv: https://arxiv.org/abs/2503.12191 · HTML: https://mlsanity.com/paper/2503.12191/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 39 of its 69 references that are in the map.
- First posted on arXiv 15 Mar 2025.

## Builds on (most cited first, 30 of 39)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Segment Anything in High Quality](https://mlsanity.com/paper/2306.01567/) (2023)
- [LAVT: Language-Aware Vision Transformer for Referring Image Segmentation](https://mlsanity.com/paper/2112.02244/) (2021)
- [Sketch-based Manga Retrieval using Manga109 Dataset](https://mlsanity.com/paper/1510.04389/) (2015)
- [Segmentation from Natural Language Expressions](https://mlsanity.com/paper/1603.06180/) (2016)
- [Reviving Iterative Training with Mask Guidance for Interactive Segmentation](https://mlsanity.com/paper/2102.06583/) (2021)
- [RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Data](https://mlsanity.com/paper/2210.12634/) (2022)
- [f-BRS: Rethinking Backpropagating Refinement for Interactive Segmentation](https://mlsanity.com/paper/2001.10331/) (2020)
- [PhraseCut: Language-based Image Segmentation in the Wild](https://mlsanity.com/paper/2008.01187/) (2020)
- [FocalClick: Towards Practical Interactive Image Segmentation](https://mlsanity.com/paper/2204.02574/) (2022)
- [The Segment Anything Model (SAM) for Remote Sensing Applications: From Zero to One Shot](https://mlsanity.com/paper/2306.16623/) (2023)
- [Doodle to Search: Practical Zero-Shot Sketch-based Image Retrieval](https://mlsanity.com/paper/1904.03451/) (2019)
- [Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2312.12470/) (2023)
- [Stabilized Sparse Scaling Algorithms for Entropy Regularized Transport Problems](https://mlsanity.com/paper/1610.06519/) (2016)
- [RRSIS: Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2306.08625/) (2023)
- [Learning Statistical Texture for Semantic Segmentation](https://mlsanity.com/paper/2103.04133/) (2021)
- [LLaFS: When Large Language Models Meet Few-Shot Segmentation](https://mlsanity.com/paper/2311.16926/) (2023)
- [Continual Semantic Segmentation with Automatic Memory Sample Selection](https://mlsanity.com/paper/2304.05015/) (2023)
- [Img2CAD: Conditioned 3D CAD Model Generation from Single Image with Structured Visual Geometry](https://mlsanity.com/paper/2410.03417/) (2024)
- [SAM-Assisted Remote Sensing Imagery Semantic Segmentation with Object and Boundary Constraints](https://mlsanity.com/paper/2312.02464/) (2023)
- [Why Do Line Drawings Work? A Realism Hypothesis](https://mlsanity.com/paper/2002.06260/) (2020)
- [Cross-Modal Bidirectional Interaction Model for Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2410.08613/) (2024)
- [UniRef++: Segment Every Reference Object in Spatial and Temporal Spaces](https://mlsanity.com/paper/2312.15715/) (2023)
- [RSRefSeg: Referring Remote Sensing Image Segmentation with Foundation Models](https://mlsanity.com/paper/2501.06809/) (2025)
- [What Can Human Sketches Do for Object Detection?](https://mlsanity.com/paper/2303.15149/) (2023)
- [Sketch2Saliency: Learning to Detect Salient Objects from Human Drawings](https://mlsanity.com/paper/2303.11502/) (2023)
- [Focused and Collaborative Feedback Integration for Interactive Image Segmentation](https://mlsanity.com/paper/2303.11880/) (2023)
- [A Unified Framework with Multimodal Fine-tuning for Remote Sensing Semantic Segmentation](https://mlsanity.com/paper/2410.11160/) (2024)
- [RESMatch: Referring Expression Segmentation in a Semi-Supervised Manner](https://mlsanity.com/paper/2402.05589/) (2024)
- [Scale-wise Bidirectional Alignment Network for Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2501.00851/) (2025)

## Abstract (arXiv)

This work advances zero-shot interactive segmentation for remote sensing imagery through three key contributions. First, we propose a novel sketch-based prompting method, enabling users to intuitively outline objects, surpassing traditional point or box prompts. Second, we introduce LTL-Sensing, the first dataset pairing human sketches with remote sensing imagery, setting a benchmark for future research. Third, we present LTL-Net, a model featuring a multi-input prompting transport module tailored for freehand sketches. Extensive experiments show our approach significantly improves segmentation accuracy and robustness over state-of-the-art methods like SAM, fostering more intuitive human-AI collaboration in remote sensing analysis and enhancing its applications.