# Different Changes Require Different Reasoning: Change-Type-Specialized Experts for Robust Change Captioning

arXiv: https://arxiv.org/abs/2609.01136 · HTML: https://mlsanity.com/paper/2609.01136/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 35 of its 58 references that are in the map.
- First posted on arXiv 1 Sep 2026.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding](https://mlsanity.com/paper/2006.16668/) (2020)
- [SPICE: Semantic Propositional Image Caption Evaluation](https://mlsanity.com/paper/1607.08822/) (2016)
- [Scaling Vision with Sparse Mixture of Experts](https://mlsanity.com/paper/2106.05974/) (2021)
- [Key-Value Memory Networks for Directly Reading Documents](https://mlsanity.com/paper/1606.03126/) (2016)
- [Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answering](https://mlsanity.com/paper/1904.04357/) (2019)
- [Memory-augmented Dense Predictive Coding for Video Representation Learning](https://mlsanity.com/paper/2008.01065/) (2020)
- [Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generation](https://mlsanity.com/paper/2303.10323/) (2023)
- [Ask Me Anything: Dynamic Memory Networks for Natural Language Processing](https://mlsanity.com/paper/1506.07285/) (2015)
- [Learning to Describe Differences Between Pairs of Similar Images](https://mlsanity.com/paper/1808.10584/) (2018)
- [Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructions](https://mlsanity.com/paper/2308.04152/) (2023)
- [MANTRA: Memory Augmented Networks for Multiple Trajectory Prediction](https://mlsanity.com/paper/2006.03340/) (2020)
- [Video Prediction Recalling Long-term Motion Context via Memory Alignment Learning](https://mlsanity.com/paper/2104.00924/) (2021)
- [CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Experts](https://mlsanity.com/paper/2405.05949/) (2024)
- [Expressing Visual Relationships via Language](https://mlsanity.com/paper/1906.07689/) (2019)
- [Describing and Localizing Multiple Changes with Transformers](https://mlsanity.com/paper/2103.14146/) (2021)
- [Structure Inference Net: Object Detection Using Scene-Level Context and Instance-Level Relationships](https://mlsanity.com/paper/1807.00119/) (2018)
- [Image Difference Captioning with Pre-training and Contrastive Learning](https://mlsanity.com/paper/2202.04298/) (2022)
- [Finding It at Another Side: A Viewpoint-Adapted Matching Encoder for Change Captioning](https://mlsanity.com/paper/2009.14352/) (2020)
- [SPMTrack: Spatio-Temporal Parameter-Efficient Fine-Tuning with Mixture of Experts for Scalable Visual Tracking](https://mlsanity.com/paper/2503.18338/) (2025)
- [Self-supervised Cross-view Representation Reconstruction for Change Captioning](https://mlsanity.com/paper/2309.16283/) (2023)
- [Neighborhood Contrastive Transformer for Change Captioning](https://mlsanity.com/paper/2303.03171/) (2023)
- [Pixel-Level Change Detection Pseudo-Label Learning for Remote Sensing Change Captioning](https://mlsanity.com/paper/2312.15311/) (2023)
- [R3Net:Relation-embedded Representation Reconstruction Network for Change Captioning](https://mlsanity.com/paper/2110.10328/) (2021)

## Abstract (arXiv)

Change captioning is the task of generating natural language descriptions that explain the changes between a pair of images. Although different change types (e.g., color shifts, object additions) exhibit distinct visual cues and require specialized reasoning processes, existing methods often overlook these distinctions. To address this limitation, we propose Multi-Expert Diagnosis for Image Change (MEDIC), a novel framework that introduces change-type awareness by explicitly modeling change categories. MEDIC employs type-specialized memory experts that dynamically retrieve type-relevant visual patterns conditioned on the input. This design enables each expert to capture diverse variations within its change type while focusing on the most informative visual cues. By softly routing inputs across type-specialized experts and learning dedicated representations for each change category, MEDIC generates more precise and type-aware change descriptions. Extensive experiments demonstrate that the proposed MEDIC consistently outperforms existing methods across diverse and challenging datasets. The code is available at hrefhttps://github.com/VisualAIKHU/MEDICGitHub.