mlsanityData to 7 Oct 2026

FUSAR-KLIP: Towards Multimodal Foundation Models for Remote Sensing

arXiv 2509.23927 ↗First posted 28 Sep 2025cs.CVcited by 2

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Cross-modal artificial intelligence, represented by visual language models, has achieved significant success in general image understanding. However, a fundamental cognitive inconsistency exists between general visual representation and remote sensing image interpretation: remote sensing images couple topography, terrain, and spatial structure, thereby inherently requiring models to possess deep geoscientific understanding. This cognitive difference is further amplified in synthetic aperture radar (SAR) imagery: while SAR possesses irreplaceable all-weather, all-day observation capabilities, it is constrained by coherent imaging mechanisms, exhibiting significant modal heterogeneity with general images. To address this inconsistency, we propose FUSAR-KLIP, the first knowledge-guided general multimodal foundational model for SAR, along with reusable data and evaluation baselines. Specifically: (1) FUSAR-GEOVL-1M (the first large-scale SAR dataset with complete geographic projection attributes) was constructed, covering multiple satellite platforms, 120,000 images, and 135 cities; (2) Aligned structured text was generated through hierarchical cognitive thought chains, accurately encoding more than 1 million multidimensional semantic information from geomorphological environment and regional attributes to spatial relationships; (3) A self-consistent iterative optimization mechanism was designed to guide cross-modal learning with this knowledge information consistent with human cognition and physical laws in a self-supervised closed loop consisting of contrast, matching, and reconstruction; (4) A unified evaluation benchmark was established in 11 typical downstream tasks in the two major categories of vision and language, and compared with 15 mainstream foundation models.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 48 of its 86 references that are in the map. First posted on arXiv 28 Sep 2025.

Builds on

48 of 86 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025SARLANG-1M: A Benchmark for Vision-Language Modeling in SAR Image Understanding4 Apr
  2. 2024EarthDial: Turning Multi-sensory Earth Observations to Interactive Dialogues19 Dec
  3. 2024Towards Vision-Language Geo-Foundation Model: A Survey13 Jun
  4. 2024VHM: Versatile and Honest Vision Language Model for Remote Sensing Image Analysis29 Mar
  5. 2024SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model18 Jan
  6. 2023SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing20 Dec
  7. 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensing24 Nov
  8. 2023RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensing20 Jun
  9. 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensing19 Jun

Other references · 39

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  6. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  7. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  8. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  9. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
  10. 2019Cascade R-CNN: High Quality Object Detection and Instance Segmentationcited by 428
  11. 2023SpectralGPT: Spectral Remote Sensing Foundation Modelcited by 250
  12. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
27 more
  1. 2025AlphaEarth Foundations: An embedding field model for accurate and efficient global mapping from sparse label datacited by 122
  2. 2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 101
  3. 2023Self-Supervised Multimodal Learning: A Surveycited by 58
  4. 2023A New Learning Paradigm for Foundation Model-based Remote Sensing Change Detectioncited by 47
  5. 2024SARDet-100K: Towards Open-Source Benchmark and ToolKit for Large-Scale SAR Object Detectioncited by 47
  6. 2023Predicting Gradient is Better: Exploring Self-Supervised Learning for SAR ATR with a Joint-Embedding Predictive Architecturecited by 43
  7. 2024MetaEarth: A Generative Foundation Model for Global-Scale Remote Sensing Image Generationcited by 41
  8. 2024Changen2: Multi-Temporal Remote Sensing Generative Change Foundation Modelcited by 37
  9. 2024STAR: A First-Ever Dataset and A Large-Scale Benchmark for Scene Graph Generation in Large-Size Satellite Imagerycited by 31
  10. 2024Multi-Label Guided Soft Contrastive Learning for Efficient Earth Observation Pretrainingcited by 27
  11. 2024GET: Unlocking the Multi-modal Potential of CLIP for Generalized Category Discoverycited by 26
  12. 2024SARATR-X: Toward Building A Foundation Model for SAR Target Recognitioncited by 25
  13. 2025A Survey on Video Temporal Grounding with Multimodal Large Language Modelcited by 19
  14. 2023Bootstrapping Interactive Image-Text Alignment for Remote Sensing Image Captioningcited by 18
  15. 2023Learning to Holistically Detect Bridges from Large-Size VHR Remote Sensing Imagerycited by 17
  16. 2024RingMo-Aerial: An Aerial Remote Sensing Foundation Model With Affine Transformation Contrastive Learningcited by 15
  17. 2025A Causal Adjustment Module for Debiasing Scene Graph Generationcited by 14
  18. 2025ATRNet-STAR: A Large Dataset and Benchmark Towards Remote Sensing Object Recognition in the Wildcited by 12
  19. 2024RS-vHeat: Heat Conduction Guided Efficient Remote Sensing Foundation Modelcited by 11
  20. 2025A Causality-aware Paradigm for Evaluating Creativity of Multimodal Large Language Modelscited by 10
  21. 2024Generative Artificial Intelligence Meets Synthetic Aperture Radar: A Surveycited by 9
  22. 2024Multi-Spectral Remote Sensing Image Retrieval Using Geospatial Foundation Modelscited by 6
  23. 2025Fifty Years of SAR Automatic Target Recognition: The Road Forwardcited by 6
  24. 2024X-Fake: Juggling Utility Evaluation and Explanation of Simulated SAR Imagescited by 4
  25. 2024Physics-Guided Detector for SAR Airplanescited by 4
  26. 2023Enhancing Representations through Heterogeneous Self-Supervised Learningcited by 2
  27. 2024EMWaveNet: Physically Explainable Neural Network Based on Electromagnetic Propagation for SAR Target Recognitioncited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026FUSAR-R1: A Large-Scale Reasoning Model for Intelligent Interpretation of SAR Imagescited by 0
  2. 2026Earth Observation Foundation Models for Terrestrial Ecohydrology: From Representation Learning to Process Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026FUSAR-GPT : A Spatiotemporal Feature-Embedded and Two-Stage Decoupled Visual Language Model for SAR Imagery
  2. 2025SARVLM: A Vision Language Foundation Model for Semantic Understanding in SAR Imagery
  3. 2025SAR-TEXT: A Large-Scale SAR Image-Text Dataset Built with SAR-Narrator and A Progressive Learning Strategy for Downstream Tasks
  4. 2025RingMoE: Mixture-of-Modality-Experts Multi-Modal Foundation Models for Universal Remote Sensing Image Interpretation
  5. 2025EarthMind: Leveraging Cross-Sensor Data for Advanced Earth Observation Interpretation with a Unified Multimodal LLM
  6. 2025SARChat-Bench-2M: A Multi-Task Vision-Language Benchmark for SAR Image Interpretation
  7. 2026Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Tasks
  8. 2026CrossEarth-SAR: A SAR-Centric and Billion-Scale Geospatial Foundation Model for Domain Generalizable Semantic Segmentation