# REO-VLM: Transforming VLM to Meet Regression Challenges in Earth Observation arXiv: https://arxiv.org/abs/2412.16583 ยท HTML: https://mlsanity.com/paper/2412.16583/ ## Key facts - Cited by 9 papers in the map (data through 14 Oct 2026). - Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. - Builds on 25 of its 35 references that are in the map. - First posted on arXiv 21 Dec 2024. ## Placed in - Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/) ## Builds on (most cited first, 25 of 25) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023) - [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022) - [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023) - [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024) - [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021) - [RemoteCLIP: A Vision Language Foundation Model for Remote Sensing](https://mlsanity.com/paper/2306.11029/) (2023) - [GeoChat: Grounded Large Vision-Language Model for Remote Sensing](https://mlsanity.com/paper/2311.15826/) (2023) - [RSGPT: A Remote Sensing Vision Language Model and Benchmark](https://mlsanity.com/paper/2307.15266/) (2023) - [EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domain](https://mlsanity.com/paper/2401.16822/) (2024) - [RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensing](https://mlsanity.com/paper/2306.11300/) (2023) - [LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Model](https://mlsanity.com/paper/2402.02544/) (2024) - [SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing](https://mlsanity.com/paper/2312.12856/) (2023) - [SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model](https://mlsanity.com/paper/2401.09712/) (2024) - [SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understanding](https://mlsanity.com/paper/2406.10100/) (2024) - [A high-resolution canopy height model of the Earth](https://mlsanity.com/paper/2204.08322/) (2022) - [Remote Sensing Vision-Language Foundation Models without Annotations via Ground Remote Alignment](https://mlsanity.com/paper/2312.06960/) (2023) - [ClimateLearn: Benchmarking Machine Learning for Weather and Climate Modeling](https://mlsanity.com/paper/2307.01909/) (2023) - [Towards Vision-Language Geo-Foundation Model: A Survey](https://mlsanity.com/paper/2406.09385/) (2024) - [TorchSpatial: A Location Encoding Framework and Benchmark for Spatial Representation Learning](https://mlsanity.com/paper/2406.15658/) (2024) - [Large Language Models for Captioning and Retrieving Remote Sensing Images](https://mlsanity.com/paper/2402.06475/) (2024) - [Multi-modal learning for geospatial vegetation forecasting](https://mlsanity.com/paper/2303.16198/) (2023) - [AGBD: A Global-scale Biomass Dataset](https://mlsanity.com/paper/2406.04928/) (2024) - [Combining Observational Data and Language for Species Range Estimation](https://mlsanity.com/paper/2410.10931/) (2024) - [M3LEO: A Multi-Modal, Multi-Label Earth Observation Dataset Integrating Interferometric SAR and Multispectral Data](https://mlsanity.com/paper/2406.04230/) (2024) ## Built on by (most cited first, 9 of 9) - [When Large Vision-Language Model Meets Large Remote Sensing Imagery: Coarse-to-Fine Text-Guided Token Pruning](https://mlsanity.com/paper/2503.07588/) (2025) - [Object Detection with Multimodal Large Vision-Language Models: An In-depth Review](https://mlsanity.com/paper/2508.19294/) (2025) - [Foundation Models in Remote Sensing: Evolving from Unimodality to Multimodality](https://mlsanity.com/paper/2603.00988/) (2026) - [LiDAR Remote Sensing Meets Weak Supervision: Concepts, Methods, and Perspectives](https://mlsanity.com/paper/2503.18384/) (2025) - [OSMDA: OpenStreetMap-based Domain Adaptation for Remote Sensing VLMs](https://mlsanity.com/paper/2603.11804/) (2026) - [AirSpatialBot: A Spatially-Aware Aerial Agent for Fine-Grained Vehicle Attribute Recognization and Retrieval](https://mlsanity.com/paper/2601.01416/) (2026) - [RemoteShield: Enable Robust Multimodal Large Language Models for Earth Observation](https://mlsanity.com/paper/2604.17243/) (2026) - [Robust Onion: Peeling Open Vocab Object Detectors Under Noise](https://mlsanity.com/paper/2606.26734/) (2026) - [Do Foundation Models Know Geometry? Probing Frozen Features for Continuous Physical Measurement](https://mlsanity.com/paper/2603.06459/) (2026) ## Abstract (arXiv) The rapid evolution of Vision Language Models (VLMs) has catalyzed significant advancements in artificial intelligence, expanding research across various disciplines, including Earth Observation (EO). While VLMs have enhanced image understanding and data processing within EO, their applications have predominantly focused on image content description. This limited focus overlooks their potential in geographic and scientific regression tasks, which are essential for diverse EO applications. To bridge this gap, this paper introduces a novel benchmark dataset, called REO-Instruct to unify regression and generation tasks specifically for the EO domain. Comprising 1.6 million multimodal EO imagery and language pairs, this dataset is designed to support both biomass regression and image content interpretation tasks. Leveraging this dataset, we develop REO-VLM, a groundbreaking model that seamlessly integrates regression capabilities with traditional generative functions. By utilizing language-driven reasoning to incorporate scientific domain knowledge, REO-VLM goes beyond solely relying on EO imagery, enabling comprehensive interpretation of complex scientific attributes from EO data. This approach establishes new performance benchmarks and significantly enhances the capabilities of environmental monitoring and resource management.