# Themes Informed Audio-visual Correspondence Learning

arXiv: https://arxiv.org/abs/2009.06573 · HTML: https://mlsanity.com/paper/2009.06573/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2020.

## Built on by (most cited first, 2 of 2)

- [A Comprehensive Survey on Video Saliency Detection with Auditory Information: the Audio-visual Consistency Perceptual is the Key!](https://mlsanity.com/paper/2206.13390/) (2022)
- [ChMusic: A Traditional Chinese Music Dataset for Evaluation of Instrument Recognition](https://mlsanity.com/paper/2108.08470/) (2021)

## Abstract (arXiv)

The applications of short-term user-generated video (UGV), such as Snapchat, and Youtube short-term videos, booms recently, raising lots of multimodal machine learning tasks. Among them, learning the correspondence between audio and visual information from videos is a challenging one. Most previous work of the audio-visual correspondence(AVC) learning only investigated constrained videos or simple settings, which may not fit the application of UGV. In this paper, we proposed new principles for AVC and introduced a new framework to set sight of videos' themes to facilitate AVC learning. We also released the KWAI-AD-AudVis corpus which contained 85432 short advertisement videos (around 913 hours) made by users. We evaluated our proposed approach on this corpus, and it was able to outperform the baseline by 23.15% absolute difference.