# Text-Guided Coarse-to-Fine Fusion Network for Robust Remote Sensing Visual Question Answering

arXiv: https://arxiv.org/abs/2411.15770 · HTML: https://mlsanity.com/paper/2411.15770/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 51 references that are in the map.
- First posted on arXiv 24 Nov 2024.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification](https://mlsanity.com/paper/1709.00029/) (2017)
- [Deep Modular Co-Attention Networks for Visual Question Answering](https://mlsanity.com/paper/1906.10770/) (2019)
- [Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Concepts](https://mlsanity.com/paper/2111.08276/) (2021)
- [RSVQA: Visual Question Answering for Remote Sensing Data](https://mlsanity.com/paper/2003.07333/) (2020)
- [FloodNet: A High Resolution Aerial Imagery Dataset for Post Flood Scene Understanding](https://mlsanity.com/paper/2012.02951/) (2020)
- [Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question Answering](https://mlsanity.com/paper/1505.05612/) (2015)
- [Where To Look: Focus Regions for Visual Question Answering](https://mlsanity.com/paper/1511.07394/) (2015)
- [Ask Your Neurons: A Neural-based Approach to Answering Questions about Images](https://mlsanity.com/paper/1505.01121/) (2015)
- [Ask Me Anything: Free-form Visual Question Answering Based on Knowledge from External Sources](https://mlsanity.com/paper/1511.06973/) (2015)
- [From Easy to Hard: Learning Language-guided Curriculum for Visual Question Answering on Remote Sensing Data](https://mlsanity.com/paper/2205.03147/) (2022)
- [Multi-Sensor Data Fusion for Cloud Removal in Global and All-Season Sentinel-2 Imagery](https://mlsanity.com/paper/2009.07683/) (2020)
- [Change Detection Meets Visual Question Answering](https://mlsanity.com/paper/2112.06343/) (2021)
- [The QXS-SAROPT Dataset for Deep Learning in SAR-Optical Data Fusion](https://mlsanity.com/paper/2103.08259/) (2021)
- [HRVQA: A Visual Question Answering Benchmark for High-Resolution Aerial Images](https://mlsanity.com/paper/2301.09460/) (2023)
- [Identifying Corresponding Patches in SAR and Optical Images with a Pseudo-Siamese CNN](https://mlsanity.com/paper/1801.08467/) (2018)
- [Multi-Modal Fusion Transformer for Visual Question Answering in Remote Sensing](https://mlsanity.com/paper/2210.04510/) (2022)
- [RSAdapter: Adapting Multimodal Models for Remote Sensing Visual Question Answering](https://mlsanity.com/paper/2310.13120/) (2023)
- [Reciprocal Translation between SAR and Optical Remote Sensing Images with Cascaded-Residual Adversarial Networks](https://mlsanity.com/paper/1901.08236/) (2019)
- [Multi-temporal Sentinel-1 and -2 Data Fusion for Optical Image Simulation](https://mlsanity.com/paper/1807.09954/) (2018)

## Built on by (most cited first, 5 of 5)

- [SARMAE: Masked Autoencoder for SAR Representation Learning](https://mlsanity.com/paper/2512.16635/) (2025)
- [Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Tasks](https://mlsanity.com/paper/2606.10819/) (2026)
- [SAR Strikes Back: A New Hope for RSVQA](https://mlsanity.com/paper/2501.08131/) (2025)
- [UAV traffic scene understanding: A regulation embedded multi-modal network and a unified benchmark](https://mlsanity.com/paper/2603.10722/) (2026)
- [CLAIRE: A Dual Encoder Network with RIFT Loss and Phi-3 Small Language Model Based Interpretability for Cross-Modality Synthetic Aperture Radar and Optical Land Cover Segmentation](https://mlsanity.com/paper/2509.11952/) (2025)

## Abstract (arXiv)

Remote Sensing Visual Question Answering (RSVQA) has gained significant research interest. However, current RSVQA methods are limited by the imaging mechanisms of optical sensors, particularly under challenging conditions such as cloud-covered and low-light scenarios. Given the all-time and all-weather imaging capabilities of Synthetic Aperture Radar (SAR), it is crucial to investigate the integration of optical-SAR images to improve RSVQA performance. In this work, we propose a Text-guided Coarse-to-Fine Fusion Network (TGFNet), which leverages the semantic relationships between question text and multi-source images to guide the network toward complementary fusion at the feature level. Specifically, we develop a Text-guided Coarse-to-Fine Attention Refinement (CFAR) module to focus on key areas related to the question in complex remote sensing images. This module progressively directs attention from broad areas to finer details through key region routing, enhancing the model's ability to focus on relevant regions. Furthermore, we propose an Adaptive Multi-Expert Fusion (AMEF) module that dynamically integrates different experts, enabling the adaptive fusion of optical and SAR features. In addition, we create the first large-scale benchmark dataset for evaluating optical-SAR RSVQA methods, comprising 6,008 well-aligned optical-SAR image pairs and 1,036,694 well-labeled question-answer pairs across 16 diverse question types, including complex relational reasoning questions. Extensive experiments on the proposed dataset demonstrate that our TGFNet effectively integrates complementary information between optical and SAR images, significantly improving the model's performance in challenging scenarios. The dataset is available at: https://github.com/mmic-lcl/. Index Terms: Remote Sensing Visual Question Answering, Multi-source Data Fusion, Multimodal, Remote Sensing, OPT-SAR.