# Taming Modality Entanglement in Continual Audio-Visual Segmentation

arXiv: https://arxiv.org/abs/2510.17234 · HTML: https://mlsanity.com/paper/2510.17234/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 37 references that are in the map.
- First posted on arXiv 20 Oct 2025.

## Builds on (most cited first, 1 of 1)

- [DF Louvain: Fast Incrementally Expanding Approach for Community Detection on Dynamic Graphs](https://mlsanity.com/paper/2404.19634/) (2024)

## Built on by (most cited first, 3 of 3)

- [Can You Hear, Localize, and Segment Continually? An Exemplar-Free Continual Learning Benchmark for Audio-Visual Segmentation](https://mlsanity.com/paper/2603.08967/) (2026)
- [Listen, Look, and Learn: Learning Without Forgetting through SAM-Audio](https://mlsanity.com/paper/2606.10887/) (2026)
- [DICS: Exploring Data Intrinsic Consistency for Visual Instruction Selection](https://mlsanity.com/paper/2608.30209/) (2026)

## Abstract (arXiv)

Recently, significant progress has been made in multi-modal continual learning, aiming to learn new tasks sequentially in multi-modal settings while preserving performance on previously learned ones. However, existing methods mainly focus on coarse-grained tasks, with limitations in addressing modality entanglement in fine-grained continual learning settings. To bridge this gap, we introduce a novel Continual Audio-Visual Segmentation (CAVS) task, aiming to continuously segment new classes guided by audio. Through comprehensive analysis, two critical challenges are identified: 1) multi-modal semantic drift, where a sounding objects is labeled as background in sequential tasks; 2) co-occurrence confusion, where frequent co-occurring classes tend to be confused. In this work, a Collision-based Multi-modal Rehearsal (CMR) framework is designed to address these challenges. Specifically, for multi-modal semantic drift, a Multi-modal Sample Selection (MSS) strategy is proposed to select samples with high modal consistency for rehearsal. Meanwhile, for co-occurence confusion, a Collision-based Sample Rehearsal (CSR) mechanism is designed, allowing for the increase of rehearsal sample frequency of those confusable classes during training process. Moreover, we construct three audio-visual incremental scenarios to verify effectiveness of our method. Comprehensive experiments demonstrate that our method significantly outperforms single-modal continual learning methods. Code can be seen at https://github.com/cqu-student/CAVS-CMR.