TerraAgent: Demonstration-Grounded Multi-Agent Reasoning for Long-Context Remote Sensing Visual Question Answering
On the map: LLM reasoning → Multimodal reasoning also fits: Spatial and embodied reasoning
also in 2 more
Abstract (arXiv)
Remote sensing visual question answering (RS-VQA) requires answers that truly depend on image evidence, yet current remote sensing multimodal large language models (RS MLLMs) are still evaluated mainly by accuracy, which cannot reveal whether a correct answer was produced without using the image. This setting introduces three challenges: demonstrations retrieved across different ground sample distances (GSD), sensors, and city textures may be poorly matched; real queries often involve long visual contexts from multi-scale tiles, multi-temporal frames, and retrieved knowledge, where evidence can be diluted and errors can propagate through multi-step reasoning; and prior-only answering remains invisible to standard metrics. We present AVA, a training-free framework that wraps a frozen RS MLLM with three modules. GDR performs GSD-bucketed, scene-graph-based demonstration retrieval and filters demonstrations whose answers are unsupported by their images. MACE decomposes each query into an evidence-typed sub-question DAG, routes sub-questions to suitable perception tools, and records verified provenance in a shared evidence ledger with bounded re-perception. CVA audits the final answer through counterfactual perturbations of cited evidence, including ablation and swap, and computes a visual-dependency score for confidence calibration and evidence-forcing re-query. On RSVQA-LR/HR, VRSBench, and LHRS-Bench, AVA improves average accuracy by up to 2.40 points over the strongest same-setting baselines, raises the visual-dependency score from 0.28 to 0.49, reduces the prior-only answer rate from 46.2 percent to 21.7 percent, and lowers the hallucination rate from 17.1 percent to 9.8 percent, with 2.6x the cost of a single backbone forward pass and no fine-tuning.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 28 of its 36 references that are in the map. First posted on arXiv 7 Oct 2026.
Builds on
28 of 36 references.
Closest ancestors · 1
Same direction, newest first
Other references · 27
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
- 2023Reasoning with Language Model is Planning with World Modelcited by 861
- 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
- 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
- 2024Mini-Gemini: Mining the Potential of Multi-modality Vision Language Modelscited by 365
- 2022Reasoning with Language Model Prompting: A Surveycited by 252
- 2022MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Textcited by 220
- 2020RSVQA: Visual Question Answering for Remote Sensing Datacited by 203
- 2024LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Modelcited by 138
- 2024Visual In-Context Learning for Large Vision-Language Modelscited by 128
- 2024VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understandingcited by 121
- 2022Retrieval Augmented Visual Question Answering with Outside Knowledgecited by 94
15 more
- 2025MAM: Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis via Role-Specialized Collaborationcited by 40
- 2023Retrieval-augmented Image Captioningcited by 29
- 2021Distilled Dual-Encoder Model for Vision-Language Understandingcited by 22
- 2024GATEAU: Selecting Influential Samples for Long Context Alignmentcited by 20
- 2025A Goal Without a Plan Is Just a Wish: Efficient and Effective Global Planner Training for Long-Horizon Agent Taskscited by 19
- 2021Long Context Question Answering via Supervised Contrastive Learningcited by 17
- 2023RSAdapter: Adapting Multimodal Models for Remote Sensing Visual Question Answeringcited by 12
- 2025Teaching Large Language Models to Maintain Contextual Faithfulness via Synthetic Tasks and Reinforcement Learningcited by 11
- 2025Remote Sensing Retrieval-Augmented Generation: Bridging Remote Sensing Imagery and Comprehensive Knowledge with a Multi-Modal Dataset and Retrieval-Augmented Generation Modelcited by 6
- 2026AVION: Aerial Vision-Language Instruction from Offline Teacher to Prompt-Tuned Networkcited by 1
- 2026Decoding the Delta: Unifying Remote Sensing Change Detection and Understanding with Multimodal Large Language Modelscited by 1
- 2026RSRCC: A Remote Sensing Regional Change Comprehension Benchmark Constructed via Retrieval-Augmented Best-of-N Rankingcited by 1
- 2026Geospatial-Temporal Sensemaking of Remote Sensing Activity Detections with Multimodal Large Language Modelcited by 1
- 2026Advancing MLLM-based UAV Image Understanding and Reasoning: A Benchmark and a Training-Free Multi-Agent Systemcited by 1
- 2026DamageScope: Vision-Language Retrieval at Scale for Disaster Damage Assessment from Satellite Imagerycited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Checkmate: interpretable and explainable RSVQA is the endgame
- 2026A Unified Framework for Efficient Remote Sensing Visual Question Answering: Adapting Dual, Hybrid, and Encoder-Decoder Architectures
- 2024Large Vision-Language Models for Remote Sensing Visual Question Answering
- 2026ReVA: A Scene-Centric Dataset Beyond Repetition for Remote Sensing Video Question Answering
- 2025Visual Reasoning Agent: Robust Vision Systems in Remote Sensing via Inference-Time Scaling
- 2025RSVLM-QA: A Benchmark Dataset for Remote Sensing Vision Language Model-based Question Answering
- 2026Selective Tool Use for Agentic Change Visual Question Answering in Remote Sensing
- 2026RS-OPSD: Reliable Privileged On-Policy-Self-Distillation for Ultra-High-Resolution Remote Sensing VQA