# Semantic-CC: Boosting Remote Sensing Image Change Captioning via Foundational Knowledge and Semantic Guidance

arXiv: https://arxiv.org/abs/2407.14032 · HTML: https://mlsanity.com/paper/2407.14032/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 93 references that are in the map.
- First posted on arXiv 19 Jul 2024.

## Builds on (most cited first, 30 of 50)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [Exploring Plain Vision Transformer Backbones for Object Detection](https://mlsanity.com/paper/2203.16527/) (2022)
- [End-to-End Multi-Task Learning with Attention](https://mlsanity.com/paper/1803.10704/) (2018)
- [Which Tasks Should Be Learned Together in Multi-task Learning?](https://mlsanity.com/paper/1905.07553/) (2019)
- [RemoteCLIP: A Vision Language Foundation Model for Remote Sensing](https://mlsanity.com/paper/2306.11029/) (2023)
- [GeoChat: Grounded Large Vision-Language Model for Remote Sensing](https://mlsanity.com/paper/2311.15826/) (2023)
- [Fully Convolutional Siamese Networks for Change Detection](https://mlsanity.com/paper/1810.08462/) (2018)
- [Remote Sensing Image Change Detection with Transformers](https://mlsanity.com/paper/2103.00208/) (2021)
- [A Transformer-Based Siamese Network for Change Detection](https://mlsanity.com/paper/2201.01293/) (2022)
- [RSGPT: A Remote Sensing Vision Language Model and Benchmark](https://mlsanity.com/paper/2307.15266/) (2023)
- [RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation based on Visual Foundation Model](https://mlsanity.com/paper/2306.16269/) (2023)
- [RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensing](https://mlsanity.com/paper/2306.11300/) (2023)
- [Advancing Plain Vision Transformer Towards Remote Sensing Foundation Model](https://mlsanity.com/paper/2208.03987/) (2022)
- [Recurrent Neural Network for Text Classification with Multi-Task Learning](https://mlsanity.com/paper/1605.05101/) (2016)
- [Robust Learning Through Cross-Task Consistency](https://mlsanity.com/paper/2006.04096/) (2020)
- [RSMamba: Remote Sensing Image Classification with State Space Model](https://mlsanity.com/paper/2403.19654/) (2024)

## Built on by (most cited first, 17 of 17)

- [Remote Sensing SpatioTemporal Vision-Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2412.02573/) (2024)
- [Semantic-CD: Remote Sensing Image Semantic Change Detection towards Open-vocabulary Setting](https://mlsanity.com/paper/2501.06808/) (2025)
- [CCExpert: Advancing MLLM Capability in Remote Sensing Change Captioning with Difference-Aware Integration and a Foundational Dataset](https://mlsanity.com/paper/2411.11360/) (2024)
- [OmniDiff: A Comprehensive Benchmark for Fine-grained Image Difference Captioning](https://mlsanity.com/paper/2503.11093/) (2025)
- [Robust Change Captioning in Remote Sensing: SECOND-CC Dataset and MModalCC Framework](https://mlsanity.com/paper/2501.10075/) (2025)
- [DeltaVLM: Interactive Remote Sensing Image Change Analysis via Instruction-guided Difference Perception](https://mlsanity.com/paper/2507.22346/) (2025)
- [MV-CC: Mask Enhanced Video Model for Remote Sensing Change Caption](https://mlsanity.com/paper/2410.23946/) (2024)
- [SAM Guided Semantic and Motion Changed Region Mining for Remote Sensing Change Captioning](https://mlsanity.com/paper/2511.21420/) (2025)
- [MONITRS: Multimodal Observations of Natural Incidents Through Remote Sensing](https://mlsanity.com/paper/2507.16228/) (2025)
- [RSICCLLM: A Multimodal Large Language Model for Remote Sensing Image Change Captioning](https://mlsanity.com/paper/2606.28266/) (2026)
- [UAV as Urban Construction Change Monitor: A New Benchmark and Change Captioning Model](https://mlsanity.com/paper/2605.04409/) (2026)
- [Forest-Chat: Adapting Vision-Language Agents for Interactive Forest Change Analysis](https://mlsanity.com/paper/2601.14637/) (2026)
- [HiSem: Hierarchical Semantic Disentangling for Remote Sensing Image Change Captioning](https://mlsanity.com/paper/2605.15024/) (2026)
- [Beyond Pixel Diffs: Benchmarking Image Change Captioning for Web UI Visual Regression Testing](https://mlsanity.com/paper/2607.01728/) (2026)
- [LBTCap: A Lightweight Bilateral Transformer for Real-Time Remote Sensing Image Change Captioning](https://mlsanity.com/paper/2607.03320/) (2026)
- [HIMEC: Directional Change Representation and Fixed-Interface Decoding for Remote Sensing Image Change Captioning](https://mlsanity.com/paper/2608.12502/) (2026)
- [MGRL-RSCC: Multi-Granularity Reward Reinforcement Learning for Fine-Grained Remote Sensing Change Captioning](https://mlsanity.com/paper/2609.26166/) (2026)

## Abstract (arXiv)

Remote sensing image change captioning (RSICC) aims to articulate the changes in objects of interest within bi-temporal remote sensing images using natural language. Given the limitations of current RSICC methods in expressing general features across multi-temporal and spatial scenarios, and their deficiency in providing granular, robust, and precise change descriptions, we introduce a novel change captioning (CC) method based on the foundational knowledge and semantic guidance, which we term Semantic-CC. Semantic-CC alleviates the dependency of high-generalization algorithms on extensive annotations by harnessing the latent knowledge of foundation models, and it generates more comprehensive and accurate change descriptions guided by pixel-level semantics from change detection (CD). Specifically, we propose a bi-temporal SAM-based encoder for dual-image feature extraction; a multi-task semantic aggregation neck for facilitating information interaction between heterogeneous tasks; a straightforward multi-scale change detection decoder to provide pixel-level semantic guidance; and a change caption decoder based on the large language model (LLM) to generate change description sentences. Moreover, to ensure the stability of the joint training of CD and CC, we propose a three-stage training strategy that supervises different tasks at various stages. We validate the proposed method on the LEVIR-CC and LEVIR-CD datasets. The experimental results corroborate the complementarity of CD and CC, demonstrating that Semantic-CC can generate more accurate change descriptions and achieve optimal performance across both tasks.