# Audio-Visual Class-Incremental Learning

arXiv: https://arxiv.org/abs/2308.11073 · HTML: https://mlsanity.com/paper/2308.11073/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 79 of its 102 references that are in the map.
- First posted on arXiv 21 Aug 2023.

## Builds on (most cited first, 30 of 79)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence](https://mlsanity.com/paper/1801.10112/) (2018)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [Lifelong Learning with Dynamically Expandable Networks](https://mlsanity.com/paper/1708.01547/) (2017)
- [VGGSound: A Large-scale Audio-Visual Dataset](https://mlsanity.com/paper/2004.14368/) (2020)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Conditional Image Synthesis With Auxiliary Classifier GANs](https://mlsanity.com/paper/1610.09585/) (2016)
- [Look, Listen and Learn](https://mlsanity.com/paper/1705.08168/) (2017)
- [DER: Dynamically Expandable Representation for Class Incremental Learning](https://mlsanity.com/paper/2103.16788/) (2021)
- [Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting](https://mlsanity.com/paper/1904.00310/) (2019)
- [Maintaining Discrimination and Fairness in Class Incremental Learning](https://mlsanity.com/paper/1911.07053/) (2019)
- [Masked Autoencoders that Listen](https://mlsanity.com/paper/2207.06405/) (2022)
- [Learning without Memorizing](https://mlsanity.com/paper/1811.08051/) (2018)
- [FOSTER: Feature Boosting and Compression for Class-Incremental Learning](https://mlsanity.com/paper/2204.04662/) (2022)
- [Compacting, Picking and Growing for Unforgetting Continual Learning](https://mlsanity.com/paper/1910.06562/) (2019)
- [Learning to Localize Sound Source in Visual Scenes](https://mlsanity.com/paper/1803.03849/) (2018)
- [Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learning](https://mlsanity.com/paper/1904.03137/) (2019)
- [PLOP: Learning without Forgetting for Continual Semantic Segmentation](https://mlsanity.com/paper/2011.11390/) (2020)

## Built on by (most cited first, 30 of 33)

- [Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning](https://mlsanity.com/paper/2403.12030/) (2024)
- [Text-to-Audio Generation Synchronized with Videos](https://mlsanity.com/paper/2403.07938/) (2024)
- [Unveiling the Power of Audio-Visual Early Fusion Transformers with Dense Interactions through Masked Modeling](https://mlsanity.com/paper/2312.01017/) (2023)
- [Class-Incremental Grouping Network for Continual Audio-Visual Learning](https://mlsanity.com/paper/2309.05281/) (2023)
- [Weakly-Supervised Audio-Visual Segmentation](https://mlsanity.com/paper/2311.15080/) (2023)
- [Audio-visual Generalized Zero-shot Learning the Easy Way](https://mlsanity.com/paper/2407.13095/) (2024)
- [Continual Audio-Visual Sound Separation](https://mlsanity.com/paper/2411.02860/) (2024)
- [DiffGAP: A Lightweight Diffusion Module in Contrastive Space for Bridging Cross-Model Gap](https://mlsanity.com/paper/2503.12131/) (2025)
- [Modality-Inconsistent Continual Learning of Multimodal Large Language Models](https://mlsanity.com/paper/2412.13050/) (2024)
- [Answering Diverse Questions via Text Attached with Key Audio-Visual Clues](https://mlsanity.com/paper/2403.06679/) (2024)
- [Semantic Grouping Network for Audio Source Separation](https://mlsanity.com/paper/2407.03736/) (2024)
- [Learning Trimodal Relation for Audio-Visual Question Answering with Missing Modality](https://mlsanity.com/paper/2407.16171/) (2024)
- [Aligning Audio-Visual Joint Representations with an Agentic Workflow](https://mlsanity.com/paper/2410.23230/) (2024)
- [PreFM: Online Audio-Visual Event Parsing via Predictive Future Modeling](https://mlsanity.com/paper/2505.23155/) (2025)
- [Characterizing Continual Learning Scenarios and Strategies for Audio Analysis](https://mlsanity.com/paper/2407.00465/) (2024)
- [Multi-scale Multi-instance Visual Sound Localization and Segmentation](https://mlsanity.com/paper/2409.00486/) (2024)
- [Unified Video-Language Pre-training with Synchronized Audio](https://mlsanity.com/paper/2405.07202/) (2024)
- [Few-Shot Open-Set Audio Classification Using Attention Information-Fused Prototypes](https://mlsanity.com/paper/2607.01297/) (2026)
- [STELLA: Continual Audio-Video Pre-training with Spatio-Temporal Localized Alignment](https://mlsanity.com/paper/2310.08204/) (2023)
- [Distribution-Level Memory Recall for Continual Learning: Preserving Knowledge and Avoiding Confusion](https://mlsanity.com/paper/2408.02695/) (2024)
- [Multi-Modal Continual Learning via Cross-Modality Adapters and Representation Alignment with Knowledge Preservation](https://mlsanity.com/paper/2511.06723/) (2025)
- [Foley-Flow: Coordinated Video-to-Audio Generation with Masked Audio-Visual Alignment and Dynamic Conditional Flows](https://mlsanity.com/paper/2603.08126/) (2026)
- [Continual Contrastive Spoken Language Understanding](https://mlsanity.com/paper/2310.02699/) (2023)
- [ReFu: Recursive Fusion for Exemplar-Free 3D Class-Incremental Learning](https://mlsanity.com/paper/2409.12326/) (2024)
- [TAIL: Text-Audio Incremental Learning](https://mlsanity.com/paper/2503.04258/) (2025)
- [Audio-Visual Class-Incremental Learning for Fish Feeding intensity Assessment in Aquaculture](https://mlsanity.com/paper/2504.15171/) (2025)
- [MCIGLE: Multimodal Exemplar-Free Class-Incremental Graph Learning](https://mlsanity.com/paper/2509.06219/) (2025)
- [Pruned Adaptation Modules: A Simple yet Strong Baseline for Continual Foundation Models](https://mlsanity.com/paper/2603.21170/) (2026)
- [FeDMRA: Federated Incremental Learning with Dynamic Memory Replay Allocation](https://mlsanity.com/paper/2603.28455/) (2026)
- [Listen, Look, and Learn: Learning Without Forgetting through SAM-Audio](https://mlsanity.com/paper/2606.10887/) (2026)

## Abstract (arXiv)

In this paper, we introduce audio-visual class-incremental learning, a class-incremental learning scenario for audio-visual video recognition. We demonstrate that joint audio-visual modeling can improve class-incremental learning, but current methods fail to preserve semantic similarity between audio and visual features as incremental step grows. Furthermore, we observe that audio-visual correlations learned in previous tasks can be forgotten as incremental steps progress, leading to poor performance. To overcome these challenges, we propose AV-CIL, which incorporates Dual-Audio-Visual Similarity Constraint (D-AVSC) to maintain both instance-aware and class-aware semantic similarity between audio-visual modalities and Visual Attention Distillation (VAD) to retain previously learned audio-guided visual attentive ability. We create three audio-visual class-incremental datasets, AVE-Class-Incremental (AVE-CI), Kinetics-Sounds-Class-Incremental (K-S-CI), and VGGSound100-Class-Incremental (VS100-CI) based on the AVE, Kinetics-Sounds, and VGGSound datasets, respectively. Our experiments on AVE-CI, K-S-CI, and VS100-CI demonstrate that AV-CIL significantly outperforms existing class-incremental learning methods in audio-visual class-incremental learning. Code and data are available at: https://github.com/weiguoPian/AV-CIL_ICCV2023.