# Text2Seg: Remote Sensing Image Semantic Segmentation via Text-Guided Visual Foundation Models

arXiv: https://arxiv.org/abs/2304.10597 · HTML: https://mlsanity.com/paper/2304.10597/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 36 references that are in the map.
- First posted on arXiv 20 Apr 2023.

## Builds on (most cited first, 9 of 9)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery](https://mlsanity.com/paper/2207.08051/) (2022)
- [LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2110.08733/) (2021)
- [UNetFormer: A UNet-like Transformer for Efficient Semantic Segmentation of Remote Sensing Urban Scene Imagery](https://mlsanity.com/paper/2109.08937/) (2021)
- [On the Opportunities and Challenges of Foundation Models for Geospatial Artificial Intelligence](https://mlsanity.com/paper/2304.06798/) (2023)

## Built on by (most cited first, 30 of 44)

- [Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey](https://mlsanity.com/paper/2305.18703/) (2023)
- [Review of Large Vision Models and Visual Prompt Engineering](https://mlsanity.com/paper/2307.00855/) (2023)
- [A Comprehensive Survey on Segment Anything Model for Vision and Beyond](https://mlsanity.com/paper/2305.08196/) (2023)
- [A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets Prompt Engineering](https://mlsanity.com/paper/2306.06211/) (2023)
- [Segment anything, from space?](https://mlsanity.com/paper/2304.13000/) (2023)
- [Good at captioning, bad at counting: Benchmarking GPT-4V on Earth observation data](https://mlsanity.com/paper/2401.17600/) (2024)
- [SAMRS: Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model](https://mlsanity.com/paper/2305.02034/) (2023)
- [SAM-Assisted Remote Sensing Imagery Semantic Segmentation with Object and Boundary Constraints](https://mlsanity.com/paper/2312.02464/) (2023)
- [Img2Loc: Revisiting Image Geolocalization using Multi-modality Foundation Models and Image-based Retrieval-Augmented Generation](https://mlsanity.com/paper/2403.19584/) (2024)
- [Vision-Language Modeling Meets Remote Sensing: Models, Datasets and Perspectives](https://mlsanity.com/paper/2505.14361/) (2025)
- [ExPLoRA: Parameter-Efficient Extended Pre-Training to Adapt Vision Transformers under Domain Shifts](https://mlsanity.com/paper/2406.10973/) (2024)
- [On the Promises and Challenges of Multimodal Foundation Models for Geographical, Environmental, Agricultural, and Urban Planning Applications](https://mlsanity.com/paper/2312.17016/) (2023)
- [Towards Artificial General Intelligence (AGI) in the Internet of Things (IoT): Opportunities and Challenges](https://mlsanity.com/paper/2309.07438/) (2023)
- [When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence System](https://mlsanity.com/paper/2309.06799/) (2023)
- [Building Privacy-Preserving and Secure Geospatial Artificial Intelligence Foundation Models](https://mlsanity.com/paper/2309.17319/) (2023)
- [CRISP-SAM2: SAM2 with Cross-Modal Interaction and Semantic Prompting for Multi-Organ Segmentation](https://mlsanity.com/paper/2506.23121/) (2025)
- [Foundation Models in Remote Sensing: Evolving from Unimodality to Multimodality](https://mlsanity.com/paper/2603.00988/) (2026)
- [ELEV-VISION-SAM: Integrated Vision Language and Foundation Model for Automated Estimation of Building Lowest Floor Elevation](https://mlsanity.com/paper/2404.12606/) (2024)
- [BadSAM: Exploring Security Vulnerabilities of SAM via Backdoor Attacks](https://mlsanity.com/paper/2305.03289/) (2023)
- [GeoSAM: Fine-tuning SAM with Multi-Modal Prompts for Mobility Infrastructure Segmentation](https://mlsanity.com/paper/2311.11319/) (2023)
- [SAM-CP: Marrying SAM with Composable Prompts for Versatile Segmentation](https://mlsanity.com/paper/2407.16682/) (2024)
- [Bridging Semantics and Geometry: A Decoupled LVLM-SAM Framework for Reasoning Segmentation in Optical Remote Sensing](https://mlsanity.com/paper/2512.19302/) (2025)
- [Towards a multimodal framework for remote sensing image change retrieval and captioning](https://mlsanity.com/paper/2406.13424/) (2024)
- [IRSAMap:Towards Large-Scale, High-Resolution Land Cover Map Vectorization](https://mlsanity.com/paper/2508.16272/) (2025)
- [FMARS: Annotating Remote Sensing Images for Disaster Management using Foundation Models](https://mlsanity.com/paper/2405.20109/) (2024)
- [ALPS: An Auto-Labeling and Pre-training Scheme for Remote Sensing Segmentation With Segment Anything Model](https://mlsanity.com/paper/2406.10855/) (2024)
- [Segment Anything Model for automated image data annotation: empirical studies using text prompts from Grounding DINO](https://mlsanity.com/paper/2406.19057/) (2024)
- [SA-MixNet: Structure-aware Mixup and Invariance Learning for Scribble-supervised Road Extraction in Remote Sensing Images](https://mlsanity.com/paper/2403.01381/) (2024)
- [DF4LCZ: A SAM-Empowered Data Fusion Framework for Scene-Level Local Climate Zone Classification](https://mlsanity.com/paper/2403.09367/) (2024)
- [Multimodal Remote Sensing Scene Classification Using VLMs and Dual-Cross Attention Networks](https://mlsanity.com/paper/2412.02531/) (2024)

## Abstract (arXiv)

Remote sensing imagery has attracted significant attention in recent years due to its instrumental role in global environmental monitoring, land usage monitoring, and more. As image databases grow each year, performing automatic segmentation with deep learning models has gradually become the standard approach for processing the data. Despite the improved performance of current models, certain limitations remain unresolved. Firstly, training deep learning models for segmentation requires per-pixel annotations. Given the large size of datasets, only a small portion is fully annotated and ready for training. Additionally, the high intra-dataset variance in remote sensing data limits the transfer learning ability of such models. Although recently proposed generic segmentation models like SAM have shown promising results in zero-shot instance-level segmentation, adapting them to semantic segmentation is a non-trivial task. To tackle these challenges, we propose a novel method named Text2Seg for remote sensing semantic segmentation. Text2Seg overcomes the dependency on extensive annotations by employing an automatic prompt generation process using different visual foundation models (VFMs), which are trained to understand semantic information in various ways. This approach not only reduces the need for fully annotated datasets but also enhances the model's ability to generalize across diverse datasets. Evaluations on four widely adopted remote sensing datasets demonstrate that Text2Seg significantly improves zero-shot prediction performance compared to the vanilla SAM model, with relative improvements ranging from 31% to 225%. Our code is available at https://github.com/Douglas2Code/Text2Seg.