mlsanityData to 14 Oct 2026

GeoPixel: Pixel Grounding Large Multimodal Model in Remote Sensing

arXiv 2501.13925 ↗First posted 23 Jan 2025cs.CVcited by 57

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Recent advances in large multimodal models (LMMs) have recognized fine-grained grounding as an imperative factor of visual understanding and dialogue. However, the benefits of such representation in LMMs are limited to the natural image domain, and these models perform poorly for remote sensing (RS). The distinct overhead viewpoint, scale variation, and presence of small objects in high-resolution RS imagery present a unique challenge in region-level comprehension. Moreover, the development of the grounding conversation capability of LMMs within RS is hindered by the lack of granular, RS domain-specific grounded data. Addressing these limitations, we propose GeoPixel - the first end-to-end high resolution RS-LMM that supports pixel-level grounding. This capability allows fine-grained visual perception by generating interleaved masks in conversation. GeoPixel supports up to 4K HD resolution in any aspect ratio, ideal for high-precision RS image analysis. To support the grounded conversation generation (GCG) in RS imagery, we curate a visually grounded dataset GeoPixelD through a semi-automated pipeline that utilizes set-of-marks prompting and spatial priors tailored for RS data to methodically control the data generation process. GeoPixel demonstrates superior performance in pixel-level comprehension, surpassing existing LMMs in both single-target and multi-target segmentation tasks. Our methodological ablation studies validate the effectiveness of each component in the overall architecture. Our code and data will be publicly released.

Key facts. Cited by 57 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 28 of its 69 references that are in the map. First posted on arXiv 23 Jan 2025.

Builds on

28 of 69 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024EarthDial: Turning Multi-sensory Earth Observations to Interactive Dialogues19 Dec
  2. 2024SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understanding14 Jun
  3. 2024VHM: Versatile and Honest Vision Language Model for Remote Sensing Image Analysis29 Mar
  4. 2024LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Model4 Feb
  5. 2024SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model18 Jan
  6. 2023RSGPT: A Remote Sensing Vision Language Model and Benchmark28 Jul

Other references · 22

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023Qwen Technical Reportcited by 3,344
  4. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  5. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  6. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  7. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  8. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  9. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
  10. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 509
  11. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  12. 2023GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interestcited by 302
10 more
  1. 2023Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4Vcited by 299
  2. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Modelscited by 286
  3. 2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Outputcited by 195
  4. 2023LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agentscited by 193
  5. 2024TextMonkey: An OCR-Free Large Multimodal Model for Understanding Documentcited by 174
  6. 2024InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HDcited by 159
  7. 2023UReader: Universal OCR-free Visually-situated Language Understanding with Multimodal Large Language Modelcited by 143
  8. 2023BuboGPT: Enabling Visual Grounding in Multi-Modal LLMscited by 114
  9. 2023InternGPT: Solving Vision-Centric Tasks by Interacting with ChatGPT Beyond Languagecited by 96
  10. 2019Towards better Validity: Dispersion based Clustering for Unsupervised Person Re-identificationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 24; 2026: 33.

57 papers.

Sort:

By area:

  1. 2025SegEarth-R1: Geospatial Pixel Reasoning via Large Language Modelcited by 42
  2. 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontierscited by 37
  3. 2025DisasterM3: A Remote Sensing Vision-Language Dataset for Disaster Damage Assessment and Responsecited by 36
  4. 2025When Large Vision-Language Model Meets Large Remote Sensing Imagery: Coarse-to-Fine Text-Guided Token Pruningcited by 35
  5. 2025EarthMind: Leveraging Cross-Sensor Data for Advanced Earth Observation Interpretation with a Unified Multimodal LLMcited by 18
  6. 2025GeoVLM-R1: Reinforcement Fine-Tuning for Improved Remote Sensing Reasoningcited by 15
  7. 2025GeoZero: Incentivizing Reasoning from Scratch on Geospatial Scenescited by 14
  8. 2026Foundation Models in Remote Sensing: Evolving from Unimodality to Multimodalitycited by 12
  9. 2025A Benchmark for Ultra-High-Resolution Remote Sensing MLLMscited by 11
  10. 2025UniGeoSeg: Towards Unified Open-World Segmentation for Geospatial Scenescited by 10
  11. 2025SegEarth-R2: Towards Comprehensive Language-guided Segmentation for Remote Sensing Imagescited by 10
  12. 2025InstructSAM: A Training-Free Framework for Instruction-Oriented Remote Sensing Object Recognitioncited by 8
38 more
  1. 2025UniChange: Unifying Change Detection with Multimodal Large Language Modelcited by 8
  2. 2026RSGround-R1: Rethinking Remote Sensing Visual Grounding through Spatial Reasoningcited by 8
  3. 2026RemoteAgent: Bridging Vague Human Intents and Earth Observation with RL-based Agentic MLLMscited by 8
  4. 2026VLRS-Bench: A Vision-Language Reasoning Benchmark for Remote Sensingcited by 6
  5. 2025VectorSynth: Fine-Grained Satellite Image Synthesis with Structured Semanticscited by 5
  6. 2026UAVBench and UAVIT-1M: Benchmarking and Enhancing MLLMs for Low-Altitude UAV Vision-Language Understandingcited by 5
  7. 2025GRASP: Geospatial pixel Reasoning viA Structured Policy learningcited by 4
  8. 2026DVGBench: Implicit-to-Explicit Visual Grounding Benchmark in UAV Imagery with Large Vision-Language Modelscited by 4
  9. 2025EcoWikiRS: Learning Ecological Representation of Satellite Images from Weak Supervision with Species Observations and Wikipediacited by 3
  10. 2025SPEX: A Vision-Language Model for Land Cover Extraction on Spectral Remote Sensing Imagescited by 3
  11. 2025Any-Optical-Model: A Universal Foundation Model for Optical Remote Sensingcited by 3
  12. 2025FUSE-RSVLM: Feature Fusion Vision-Language Model for Remote Sensingcited by 3
  13. 2026ProtoFlow: Mitigating Forgetting in Class-Incremental Remote Sensing Segmentation via Low-Curvature Prototype Flowcited by 3
  14. 2025PathMR: Multimodal Visual Reasoning for Interpretable Pathology Diagnosiscited by 2
  15. 2026Enabling Training-Free Text-Based Remote Sensing Segmentationcited by 2
  16. 2026GeoAlignCLIP: Enhancing Fine-Grained Vision-Language Alignment in Remote Sensing via Multi-Granular Consistency Learningcited by 2
  17. 2026WalkGPT: Grounded Vision-Language Conversation with Depth-Aware Segmentation for Pedestrian Navigationcited by 2
  18. 2026GeoMeld: Toward Semantically Grounded Foundation Models for Remote Sensingcited by 2
  19. 2026Agentic AI for Remote Sensing: Technical Challenges and Research Directionscited by 2
  20. 2026WeaveEarth: Structured Evidence Construction and Reasoning for Training-Free UHR Remote Sensing Understandingcited by 2
  21. 2025SAR Object Detection with Self-Supervised Pretraining and Curriculum-Aware Samplingcited by 1
  22. 2025Chain-of-Talkers (CoTalk): Fast Human Annotation of Dense Image Captionscited by 1
  23. 2026GeoHeight-Bench: Towards Height-Aware Multimodal Reasoning in Remote Sensingcited by 1
  24. 2026AI and Open-data Driven Scalable Solar Power Profilingcited by 1
  25. 2026UHR-Micro: Diagnosing and Mitigating the Resolution Illusion in Earth Observation VLMscited by 1
  26. 2026B-GRTO: Bootstrapped Group Relative Tool Optimization for Referring Segmentationcited by 1
  27. 2026An Open-Source Benchmark and Baseline for Multi-temporal Referring Segmentationcited by 1
  28. 2026GeoSearcher: Anchor-Guided Progressive Reasoning for Remote Sensing Visual Grounding with Process Supervisioncited by 1
  29. 2026CROSS: Cascaded Distillation and Dual-Constraint Grounding for Remote Sensing Referring Segmentationcited by 1
  30. 2025LandSegmenter: Towards a Flexible Foundation Model for Land Use and Land Cover Mappingcited by 0
  31. 2025Cross-modal Context-aware Learning for Visual Prompt Guided Multimodal Image Understanding in Remote Sensingcited by 0
  32. 2025MMLANDMARKS: a Cross-View Instance-Level Benchmark for Geo-Spatial Understandingcited by 0
  33. 2026GroundSet: A Cadastral-Grounded Dataset for Spatial Understanding with Vector Datacited by 0
  34. 2026Evaluating Remote Sensing Image Captions Beyond Metric Biasescited by 0
  35. 2026WOW-Seg: A Word-free Open World Segmentation Modelcited by 0
  36. 2026SkyNative: A Native Multimodal Architecture for Remote Sensing Vision-Language Understandingcited by 0
  37. 2026Beyond Appearance: Can Multimodal Large Language Models Exploit Vertical Structure for Remote Sensing Natural Scene Understanding?cited by 0
  38. 2026Geometric Coastline Localization using Vision-Language Modelscited by 0

50 most cited of 57 All 57 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensing
  2. 2025GeoPix: Multi-Modal Large Language Model for Pixel-level Image Understanding in Remote Sensing
  3. 2025GeoViS: Geospatially Rewarded Visual Search for Remote Sensing Visual Grounding
  4. 2024GeoGround: A Unified Large Vision-Language Model for Remote Sensing Visual Grounding
  5. 2025GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolution
  6. 2026Multimodal Large Language Models for Remote Sensing Image Understanding: Domain-Specific or General-Purpose?
  7. 2023GLaMM: Pixel Grounding Large Multimodal Model
  8. 2025SATGround: A Spatially-Aware Approach for Visual Grounding in Remote Sensing