# Curriculum Audiovisual Learning

arXiv: https://arxiv.org/abs/2001.09414 · HTML: https://mlsanity.com/paper/2001.09414/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jan 2020.

## Built on by (most cited first, 14 of 14)

- [Curriculum Learning: A Survey](https://mlsanity.com/paper/2101.10382/) (2021)
- [Cyclic Co-Learning of Sounding Object Visual Grounding and Sound Separation](https://mlsanity.com/paper/2104.02026/) (2021)
- [Class-aware Sounding Objects Localization via Audiovisual Correspondence](https://mlsanity.com/paper/2112.11749/) (2021)
- [Self-Supervised Predictive Learning: A Negative-Free Method for Sound Source Localization in Visual Scenes](https://mlsanity.com/paper/2203.13412/) (2022)
- [Self-supervised object detection from audio-visual correspondence](https://mlsanity.com/paper/2104.06401/) (2021)
- [Unsupervised Sound Localization via Iterative Contrastive Learning](https://mlsanity.com/paper/2104.00315/) (2021)
- [Audio-visual Representation Learning for Anomaly Events Detection in Crowds](https://mlsanity.com/paper/2110.14862/) (2021)
- [Look, Listen, and Attack: Backdoor Attacks Against Video Action Recognition](https://mlsanity.com/paper/2301.00986/) (2023)
- [Visually-Guided Sound Source Separation with Audio-Visual Predictive Coding](https://mlsanity.com/paper/2306.10684/) (2023)
- [A Survey of Recent Advances and Challenges in Deep Audio-Visual Correlation Learning](https://mlsanity.com/paper/2412.00049/) (2024)
- [Statistical Measures For Defining Curriculum Scoring Function](https://mlsanity.com/paper/2103.00147/) (2021)
- [Video-to-Music Recommendation using Temporal Alignment of Segments](https://mlsanity.com/paper/2306.07187/) (2023)
- [Recent Advances and Challenges in Deep Audio-Visual Correlation Learning](https://mlsanity.com/paper/2202.13673/) (2022)
- [AudioScene: Integrating Object-Event Audio into 3D Scenes](https://mlsanity.com/paper/2512.07845/) (2025)

## Abstract (arXiv)

Associating sound and its producer in complex audiovisual scene is a challenging task, especially when we are lack of annotated training data. In this paper, we present a flexible audiovisual model that introduces a soft-clustering module as the audio and visual content detector, and regards the pervasive property of audiovisual concurrency as the latent supervision for inferring the correlation among detected contents. To ease the difficulty of audiovisual learning, we propose a novel curriculum learning strategy that trains the model from simple to complex scene. We show that such ordered learning procedure rewards the model the merits of easy training and fast convergence. Meanwhile, our audiovisual model can also provide effective unimodal representation and cross-modal alignment performance. We further deploy the well-trained model into practical audiovisual sound localization and separation task. We show that our localization model significantly outperforms existing methods, based on which we show comparable performance in sound separation without referring external visual supervision. Our video demo can be found at https://youtu.be/kuClfGG0cFU.