# Falcon: A Remote Sensing Vision-Language Foundation Model (Technical Report)

arXiv: https://arxiv.org/abs/2503.11070 · HTML: https://mlsanity.com/paper/2503.11070/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 16 of its 103 references that are in the map.
- First posted on arXiv 14 Mar 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 16 of 16)

- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies](https://mlsanity.com/paper/2404.06395/) (2024)
- [SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Models](https://mlsanity.com/paper/2311.07575/) (2023)
- [LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2110.08733/) (2021)
- [RSGPT: A Remote Sensing Vision Language Model and Benchmark](https://mlsanity.com/paper/2307.15266/) (2023)
- [LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Model](https://mlsanity.com/paper/2402.02544/) (2024)
- [VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks](https://mlsanity.com/paper/2406.08394/) (2024)
- [SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model](https://mlsanity.com/paper/2401.09712/) (2024)
- [Exploring a Fine-Grained Multiscale Method for Cross-Modal Remote Sensing Image Retrieval](https://mlsanity.com/paper/2204.09868/) (2022)
- [Remote Sensing Vision-Language Foundation Models without Annotations via Ground Remote Alignment](https://mlsanity.com/paper/2312.06960/) (2023)
- [Remote Sensing ChatGPT: Solving Remote Sensing Tasks with ChatGPT and Visual Models](https://mlsanity.com/paper/2401.09083/) (2024)
- [Large Language Models for Captioning and Retrieving Remote Sensing Images](https://mlsanity.com/paper/2402.06475/) (2024)
- [EarthMarker: A Visual Prompting Multi-modal Large Language Model for Remote Sensing](https://mlsanity.com/paper/2407.13596/) (2024)
- [Popeye: A Unified Visual-Language Model for Multi-Source Ship Detection from Remote Sensing Imagery](https://mlsanity.com/paper/2403.03790/) (2024)
- [MGIMM: Multi-Granularity Instruction Multimodal Model for Attribute-Guided Remote Sensing Image Detailed Description](https://mlsanity.com/paper/2406.04716/) (2024)

## Built on by (most cited first, 29 of 29)

- [RemoteSAM: Towards Segment Anything for Earth Observation](https://mlsanity.com/paper/2505.18022/) (2025)
- [RemoteReasoner: Towards Unifying Geospatial Reasoning Workflow](https://mlsanity.com/paper/2507.19280/) (2025)
- [RingMoE: Mixture-of-Modality-Experts Multi-Modal Foundation Models for Universal Remote Sensing Image Interpretation](https://mlsanity.com/paper/2504.03166/) (2025)
- [REOBench: Benchmarking Robustness of Earth Observation Foundation Models](https://mlsanity.com/paper/2505.16793/) (2025)
- [A Benchmark for Ultra-High-Resolution Remote Sensing MLLMs](https://mlsanity.com/paper/2512.17319/) (2025)
- [SegEarth-R2: Towards Comprehensive Language-guided Segmentation for Remote Sensing Images](https://mlsanity.com/paper/2512.20013/) (2025)
- [RemoteAgent: Bridging Vague Human Intents and Earth Observation with RL-based Agentic MLLMs](https://mlsanity.com/paper/2604.07765/) (2026)
- [SkyMoE: A Vision-Language Foundation Model for Enhancing Geospatial Interpretation with Mixture of Experts](https://mlsanity.com/paper/2512.02517/) (2025)
- [LLaVA-UHD v3: Progressive Visual Compression for Efficient Native-Resolution Encoding in MLLMs](https://mlsanity.com/paper/2511.21150/) (2025)
- [UAVBench and UAVIT-1M: Benchmarking and Enhancing MLLMs for Low-Altitude UAV Vision-Language Understanding](https://mlsanity.com/paper/2603.14336/) (2026)
- [SAR-TEXT: A Large-Scale SAR Image-Text Dataset Built with SAR-Narrator and A Progressive Learning Strategy for Downstream Tasks](https://mlsanity.com/paper/2507.18743/) (2025)
- [Survey of Multimodal Geospatial Foundation Models: Techniques, Applications, and Challenges](https://mlsanity.com/paper/2510.22964/) (2025)
- [Co-Training Vision Language Models for Remote Sensing Multi-task Learning](https://mlsanity.com/paper/2511.21272/) (2025)
- [RemoteShield: Enable Robust Multimodal Large Language Models for Earth Observation](https://mlsanity.com/paper/2604.17243/) (2026)
- [Agentic AI for Remote Sensing: Technical Challenges and Research Directions](https://mlsanity.com/paper/2604.24919/) (2026)
- [RemoteZero: Geospatial Reasoning with Zero Labels](https://mlsanity.com/paper/2605.04451/) (2026)
- [RSICCLLM: A Multimodal Large Language Model for Remote Sensing Image Change Captioning](https://mlsanity.com/paper/2606.28266/) (2026)
- [WeaveEarth: Structured Evidence Construction and Reasoning for Training-Free UHR Remote Sensing Understanding](https://mlsanity.com/paper/2607.10120/) (2026)
- [UAV traffic scene understanding: A regulation embedded multi-modal network and a unified benchmark](https://mlsanity.com/paper/2603.10722/) (2026)
- [SITS-DECO: A Generative Decoder Is All You Need For Multitask Satellite Image Time Series Modelling](https://mlsanity.com/paper/2510.21813/) (2025)
- [LandSegmenter: Towards a Flexible Foundation Model for Land Use and Land Cover Mapping](https://mlsanity.com/paper/2511.08156/) (2025)
- [DiffuSAM: Diffusion Guided Zero-Shot Object Grounding for Remote Sensing Imagery](https://mlsanity.com/paper/2604.18201/) (2026)
- [SenseBench: A Benchmark for Remote Sensing Low-Level Visual Perception and Description in Large Vision-Language Models](https://mlsanity.com/paper/2605.10576/) (2026)
- [Text-RSIR: A Text-Guided Framework for Efficient Remote Sensing Image Transmission and Reconstruction](https://mlsanity.com/paper/2605.15558/) (2026)
- [Improving Visual Grounding in Remote Sensing via Cluster-Guided Refinement and Model Ensemble Voting](https://mlsanity.com/paper/2606.00556/) (2026)
- [More with Less: a Large Scale Remote Sensing VLM with a Simple Recipe](https://mlsanity.com/paper/2607.15942/) (2026)
- [Multimodal Large Language Models for Remote Sensing Image Understanding: Domain-Specific or General-Purpose?](https://mlsanity.com/paper/2607.20284/) (2026)
- [GeoNLI - A Natural Language Interpreter for Satellite Imagery](https://mlsanity.com/paper/2609.28741/) (2026)
- [USAI-Quant: A Quantitative Reasoning Benchmark for Vision-Language Models in Built Environments](https://mlsanity.com/paper/2609.32813/) (2026)

## Abstract (arXiv)

This paper introduces a holistic vision-language foundation model tailored for remote sensing, named Falcon. Falcon offers a unified, prompt-based paradigm that effectively executes comprehensive and complex remote sensing tasks. Falcon demonstrates powerful understanding and reasoning abilities at the image, region, and pixel levels. Specifically, given simple natural language instructions and remote sensing images, Falcon can produce impressive results in text form across 14 distinct tasks, i.e., image classification, object detection, segmentation, image captioning, and etc. To facilitate Falcon's training and empower its representation capacity to encode rich spatial and semantic information, we developed Falcon_SFT, a large-scale, multi-task, instruction-tuning dataset in the field of remote sensing. The Falcon_SFT dataset consists of approximately 78 million high-quality data samples, covering 5.6 million multi-spatial resolution and multi-view remote sensing images with diverse instructions. It features hierarchical annotations and undergoes manual sampling verification to ensure high data quality and reliability. Extensive comparative experiments are conducted, which verify that Falcon achieves remarkable performance over 67 datasets and 14 tasks, despite having only 0.7B parameters. We release the complete dataset, code, and model weights at https://github.com/TianHuiLab/Falcon, hoping to help further develop the open-source community.