AeroReformer2: Spoken-Query Referring Segmentation for Aerial Images
Not on the map. Why?
Abstract (arXiv)
Spoken language offers a natural, hands-free interface for specifying an arbitrary target in dense remote-sensing imagery, yet existing referring remote-sensing image segmentation benchmarks accept only written expressions. To bridge this gap, we introduce dataset, a spoken-query benchmark derived from RISBench that adds accent- and voice-diverse speech while preserving the original image, mask, and data splits. Its hard evaluation sets combine rotor, wind, and mixed interference with three signal-to-noise levels. We also propose model, an efficient bilateral network that combines a boundary-preserving visual path with token-preserving speech encoding, kernel linear cross-modal attention, and a resolution refinement head. The design conditions visual features at two scales without materializing a dense speech--visual affinity matrix, then restores fine boundaries using high-resolution visual features. On the clean test split, model with Swin-Base achieves 62.09% mean intersection over union (mIoU) and 68.22% overall intersection over union (oIoU), outperforming the strongest audio-adapted remote-sensing baseline by 5.38 and 2.08 percentage points, respectively. It retains the best hard-set mIoU at 54.09%. To the best of our knowledge, this is the first benchmark and model study of full-sentence spoken-query referring segmentation for remote-sensing imagery. The code will be made publicly available.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 43 of its 57 references that are in the map. First posted on arXiv 9 Aug 2026.
Builds on
43 of 57 references.
References · 43
- 2017Attention Is All You Needcited by 60,361
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
- 2023Segment Anythingcited by 8,234
- 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
- 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
- 2019Searching for MobileNetV3cited by 1,994
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
- 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
- 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
31 more
- 2017DOTA: A Large-scale Dataset for Object Detection in Aerial Imagescited by 623
- 2018Efficient Attention: Attention with Linear Complexitiescited by 368
- 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
- 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 348
- 2019Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmarkcited by 340
- 2019Cross-Modal Self-Attention Network for Referring Image Segmentationcited by 263
- 2022Audio-Visual Segmentationcited by 151
- 2020Referring Image Segmentation via Cross-Modal Progressive Comprehensioncited by 131
- 2017Recurrent Multimodal Interaction for Referring Image Segmentationcited by 125
- 2021Rethinking BiSeNet For Real-time Semantic Segmentationcited by 124
- 2024VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understandingcited by 121
- 2020Linguistic Structure Guided Context Modeling for Referring Image Segmentationcited by 102
- 2022PIDNet: A Real-time Semantic Segmentation Network Inspired by PID Controllerscited by 84
- 2023Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentationcited by 76
- 2023AVSegFormer: Audio-Visual Segmentation with Transformercited by 68
- 2023Beyond One-to-One: Rethinking the Referring Image Segmentationcited by 68
- 2023Zero-shot Referring Image Segmentation with Global-Local Context Featurescited by 61
- 2023RRSIS: Referring Remote Sensing Image Segmentationcited by 59
- 2023Annotation-free Audio-Visual Segmentationcited by 46
- 2023Mask Grounding for Referring Image Segmentationcited by 40
- 2024Ref-AVS: Refer and Segment Objects in Audio-Visual Scenescited by 33
- 2023Cooperation Does Matter: Exploring Multi-Order Bilateral Relations for Audio-Visual Segmentationcited by 26
- 2024Cross-Modal Bidirectional Interaction Model for Referring Remote Sensing Image Segmentationcited by 26
- 2024Exploring Fine-Grained Image-Text Alignment for Referring Remote Sensing Image Segmentationcited by 24
- 2025Hybrid Global-Local Representation with Augmented Spatial Guidance for Zero-Shot Referring Image Segmentationcited by 14
- 2025Intra- and Cross-Scale Feature Interaction Network for Referring Remote Sensing Image Segmentationcited by 12
- 2025Scale-wise Bidirectional Alignment Network for Referring Remote Sensing Image Segmentationcited by 11
- 2025Towards Omnimodal Expressions and Reasoning in Referring Audio-Visual Segmentationcited by 11
- 2023LSwinSR: UAV Imagery Super-Resolution based on Linear Swin Transformercited by 6
- 2025AeroReformer: Aerial Referring Transformer for UAV-based Referring Image Segmentationcited by 5
- 2025Layover or Direct Flight: Rethinking Audio-Guided Image Segmentationcited by 1
Built on by
Not cited yet.