# FUSAR-KLIP: Towards Multimodal Foundation Models for Remote Sensing arXiv: https://arxiv.org/abs/2509.23927 ยท HTML: https://mlsanity.com/paper/2509.23927/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. - Builds on 48 of its 86 references that are in the map. - First posted on arXiv 28 Sep 2025. ## Placed in - Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/) ## Builds on (most cited first, 30 of 48) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021) - [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022) - [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021) - [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021) - [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021) - [Exploring Plain Vision Transformer Backbones for Object Detection](https://mlsanity.com/paper/2203.16527/) (2022) - [Cascade R-CNN: High Quality Object Detection and Instance Segmentation](https://mlsanity.com/paper/1906.09756/) (2019) - [RemoteCLIP: A Vision Language Foundation Model for Remote Sensing](https://mlsanity.com/paper/2306.11029/) (2023) - [GeoChat: Grounded Large Vision-Language Model for Remote Sensing](https://mlsanity.com/paper/2311.15826/) (2023) - [SpectralGPT: Spectral Remote Sensing Foundation Model](https://mlsanity.com/paper/2311.07113/) (2023) - [Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond](https://mlsanity.com/paper/2101.11517/) (2021) - [RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensing](https://mlsanity.com/paper/2306.11300/) (2023) - [AlphaEarth Foundations: An embedding field model for accurate and efficient global mapping from sparse label data](https://mlsanity.com/paper/2507.22291/) (2025) - [SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing](https://mlsanity.com/paper/2312.12856/) (2023) - [SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model](https://mlsanity.com/paper/2401.09712/) (2024) - [Graph Foundation Models: Concepts, Opportunities and Challenges](https://mlsanity.com/paper/2310.11829/) (2023) - [VHM: Versatile and Honest Vision Language Model for Remote Sensing Image Analysis](https://mlsanity.com/paper/2403.20213/) (2024) - [EarthDial: Turning Multi-sensory Earth Observations to Interactive Dialogues](https://mlsanity.com/paper/2412.15190/) (2024) - [Self-Supervised Multimodal Learning: A Survey](https://mlsanity.com/paper/2304.01008/) (2023) - [SARDet-100K: Towards Open-Source Benchmark and ToolKit for Large-Scale SAR Object Detection](https://mlsanity.com/paper/2403.06534/) (2024) - [A New Learning Paradigm for Foundation Model-based Remote Sensing Change Detection](https://mlsanity.com/paper/2312.01163/) (2023) - [Predicting Gradient is Better: Exploring Self-Supervised Learning for SAR ATR with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2311.15153/) (2023) - [MetaEarth: A Generative Foundation Model for Global-Scale Remote Sensing Image Generation](https://mlsanity.com/paper/2405.13570/) (2024) - [Changen2: Multi-Temporal Remote Sensing Generative Change Foundation Model](https://mlsanity.com/paper/2406.17998/) (2024) - [Towards Vision-Language Geo-Foundation Model: A Survey](https://mlsanity.com/paper/2406.09385/) (2024) - [STAR: A First-Ever Dataset and A Large-Scale Benchmark for Scene Graph Generation in Large-Size Satellite Imagery](https://mlsanity.com/paper/2406.09410/) (2024) - [Multi-Label Guided Soft Contrastive Learning for Efficient Earth Observation Pretraining](https://mlsanity.com/paper/2405.20462/) (2024) ## Built on by (most cited first, 3 of 3) - [FUSAR-R1: A Large-Scale Reasoning Model for Intelligent Interpretation of SAR Images](https://mlsanity.com/paper/2607.16819/) (2026) - [SARATR-X-v2: Scale-Aware Structural Pre-Training for SAR Foundation Models](https://mlsanity.com/paper/2607.23238/) (2026) - [Earth Observation Foundation Models for Terrestrial Ecohydrology: From Representation Learning to Process Inference](https://mlsanity.com/paper/2608.15282/) (2026) ## Abstract (arXiv) Cross-modal artificial intelligence, represented by visual language models, has achieved significant success in general image understanding. However, a fundamental cognitive inconsistency exists between general visual representation and remote sensing image interpretation: remote sensing images couple topography, terrain, and spatial structure, thereby inherently requiring models to possess deep geoscientific understanding. This cognitive difference is further amplified in synthetic aperture radar (SAR) imagery: while SAR possesses irreplaceable all-weather, all-day observation capabilities, it is constrained by coherent imaging mechanisms, exhibiting significant modal heterogeneity with general images. To address this inconsistency, we propose FUSAR-KLIP, the first knowledge-guided general multimodal foundational model for SAR, along with reusable data and evaluation baselines. Specifically: (1) FUSAR-GEOVL-1M (the first large-scale SAR dataset with complete geographic projection attributes) was constructed, covering multiple satellite platforms, 120,000 images, and 135 cities; (2) Aligned structured text was generated through hierarchical cognitive thought chains, accurately encoding more than 1 million multidimensional semantic information from geomorphological environment and regional attributes to spatial relationships; (3) A self-consistent iterative optimization mechanism was designed to guide cross-modal learning with this knowledge information consistent with human cognition and physical laws in a self-supervised closed loop consisting of contrast, matching, and reconstruction; (4) A unified evaluation benchmark was established in 11 typical downstream tasks in the two major categories of vision and language, and compared with 15 mainstream foundation models.