# From Easy to Hard: Learning Language-guided Curriculum for Visual Question Answering on Remote Sensing Data

arXiv: https://arxiv.org/abs/2205.03147 · HTML: https://mlsanity.com/paper/2205.03147/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 55 references that are in the map.
- First posted on arXiv 6 May 2022.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [Unified Vision-Language Pre-Training for Image Captioning and VQA](https://mlsanity.com/paper/1909.11059/) (2019)
- [AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classification](https://mlsanity.com/paper/1608.05167/) (2016)
- [Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark](https://mlsanity.com/paper/1909.00133/) (2019)
- [Visual7W: Grounded Question Answering in Images](https://mlsanity.com/paper/1511.03416/) (2015)
- [Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding](https://mlsanity.com/paper/1606.01847/) (2016)
- [RSVQA: Visual Question Answering for Remote Sensing Data](https://mlsanity.com/paper/2003.07333/) (2020)
- [In Defense of Grid Features for Visual Question Answering](https://mlsanity.com/paper/2001.03615/) (2020)
- [MUTAN: Multimodal Tucker Fusion for Visual Question Answering](https://mlsanity.com/paper/1705.06676/) (2017)
- [Deep learning in remote sensing: a review](https://mlsanity.com/paper/1710.03959/) (2017)
- [Visual Question Answering: A Survey of Methods and Datasets](https://mlsanity.com/paper/1607.05910/) (2016)
- [Multi-modal Factorized Bilinear Pooling with Co-Attention Learning for Visual Question Answering](https://mlsanity.com/paper/1708.01471/) (2017)
- [A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Input](https://mlsanity.com/paper/1410.0210/) (2014)
- [Say As You Wish: Fine-grained Control of Image Caption Generation with Abstract Scene Graphs](https://mlsanity.com/paper/2003.00387/) (2020)
- [Beyond Bilinear: Generalized Multimodal Factorized High-order Pooling for Visual Question Answering](https://mlsanity.com/paper/1708.03619/) (2017)
- [A Survey on Object Detection in Optical Remote Sensing Images](https://mlsanity.com/paper/1603.06201/) (2016)
- [Tips and Tricks for Visual Question Answering: Learnings from the 2017 Challenge](https://mlsanity.com/paper/1708.02711/) (2017)
- [Where To Look: Focus Regions for Visual Question Answering](https://mlsanity.com/paper/1511.07394/) (2015)
- [Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery](https://mlsanity.com/paper/1803.02642/) (2018)
- [Learning to Count Objects in Natural Images for Visual Question Answering](https://mlsanity.com/paper/1802.05766/) (2018)
- [GETNET: A General End-to-end Two-dimensional CNN Framework for Hyperspectral Image Change Detection](https://mlsanity.com/paper/1905.01662/) (2019)
- [ABC-CNN: An Attention Based Convolutional Neural Network for Visual Question Answering](https://mlsanity.com/paper/1511.05960/) (2015)
- [Land Use Classification in Remote Sensing Images by Convolutional Neural Networks](https://mlsanity.com/paper/1508.00092/) (2015)
- [A Competence-aware Curriculum for Visual Concepts Learning via Question Answering](https://mlsanity.com/paper/2007.01499/) (2020)

## Built on by (most cited first, 30 of 39)

- [GeoChat: Grounded Large Vision-Language Model for Remote Sensing](https://mlsanity.com/paper/2311.15826/) (2023)
- [RSGPT: A Remote Sensing Vision Language Model and Benchmark](https://mlsanity.com/paper/2307.15266/) (2023)
- [RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Data](https://mlsanity.com/paper/2210.12634/) (2022)
- [VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding](https://mlsanity.com/paper/2406.12384/) (2024)
- [SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model](https://mlsanity.com/paper/2401.09712/) (2024)
- [Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performance](https://mlsanity.com/paper/2410.16261/) (2024)
- [Vision-Language Models in Remote Sensing: Current Progress and Future Trends](https://mlsanity.com/paper/2305.05726/) (2023)
- [RRSIS: Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2306.08625/) (2023)
- [RS-Agent: Automating Remote Sensing Tasks through Intelligent Agent](https://mlsanity.com/paper/2406.07089/) (2024)
- [Cross-Modal Bidirectional Interaction Model for Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2410.08613/) (2024)
- [Parameter-Efficient Transfer Learning for Remote Sensing Image-Text Retrieval](https://mlsanity.com/paper/2308.12509/) (2023)
- [ChatEarthNet: A Global-Scale Image-Text Dataset Empowering Vision-Language Geo-Foundation Models](https://mlsanity.com/paper/2402.11325/) (2024)
- [A Practitioner's Guide to Continual Multimodal Pretraining](https://mlsanity.com/paper/2408.14471/) (2024)
- [HRVQA: A Visual Question Answering Benchmark for High-Resolution Aerial Images](https://mlsanity.com/paper/2301.09460/) (2023)
- [RingMo-Agent: A Unified Remote Sensing Foundation Model for Multi-Platform and Multi-Modal Reasoning](https://mlsanity.com/paper/2507.20776/) (2025)
- [Better, Not Just More: Data-Centric Machine Learning for Earth Observation](https://mlsanity.com/paper/2312.05327/) (2023)
- [RS-MoE: A Vision-Language Model with Mixture of Experts for Remote Sensing Image Captioning and Visual Question Answering](https://mlsanity.com/paper/2411.01595/) (2024)
- [NextBestPath: Efficient 3D Mapping of Unseen Environments](https://mlsanity.com/paper/2502.05378/) (2025)
- [LISAT: Language-Instructed Segmentation Assistant for Satellite Imagery](https://mlsanity.com/paper/2505.02829/) (2025)
- [RSAdapter: Adapting Multimodal Models for Remote Sensing Visual Question Answering](https://mlsanity.com/paper/2310.13120/) (2023)
- [Multilingual Augmentation for Robust Visual Question Answering in Remote Sensing Images](https://mlsanity.com/paper/2304.03844/) (2023)
- [CromSS: Cross-modal pre-training with noisy labels for remote sensing image segmentation](https://mlsanity.com/paper/2405.01217/) (2024)
- [Diffusion Curriculum: Synthetic-to-Real Data Curriculum via Image-Guided Diffusion](https://mlsanity.com/paper/2410.13674/) (2024)
- [Text-Guided Coarse-to-Fine Fusion Network for Robust Remote Sensing Visual Question Answering](https://mlsanity.com/paper/2411.15770/) (2024)
- [The curse of language biases in remote sensing VQA: the role of spatial attributes, language diversity, and the need for clear evaluation](https://mlsanity.com/paper/2311.16782/) (2023)
- [ZeShot-VQA: Zero-Shot Visual Question Answering Framework with Answer Mapping for Natural Disaster Damage Assessment](https://mlsanity.com/paper/2506.00238/) (2025)
- [From Easy to Hard: Learning Curricular Shape-aware Features for Robust Panoptic Scene Graph Generation](https://mlsanity.com/paper/2407.09191/) (2024)
- [FrogDogNet: Fourier frequency Retained visual prompt Output Guidance for Domain Generalization of CLIP in Remote Sensing](https://mlsanity.com/paper/2504.16433/) (2025)
- [Overcoming Language Bias in Remote Sensing Visual Question Answering via Adversarial Training](https://mlsanity.com/paper/2306.00483/) (2023)
- [Dynamic Task and Weight Prioritization Curriculum Learning for Multimodal Imagery](https://mlsanity.com/paper/2310.19109/) (2023)

## Abstract (arXiv)

Visual question answering (VQA) for remote sensing scene has great potential in intelligent human-computer interaction system. Although VQA in computer vision has been widely researched, VQA for remote sensing data (RSVQA) is still in its infancy. There are two characteristics that need to be specially considered for the RSVQA task. 1) No object annotations are available in RSVQA datasets, which makes it difficult for models to exploit informative region representation; 2) There are questions with clearly different difficulty levels for each image in the RSVQA task. Directly training a model with questions in a random order may confuse the model and limit the performance. To address these two problems, in this paper, a multi-level visual feature learning method is proposed to jointly extract language-guided holistic and regional image features. Besides, a self-paced curriculum learning (SPCL)-based VQA model is developed to train networks with samples in an easy-to-hard way. To be more specific, a language-guided SPCL method with a soft weighting strategy is explored in this work. The proposed model is evaluated on three public datasets, and extensive experimental results show that the proposed RSVQA framework can achieve promising performance.