# Weakly Supervised Representation Learning for Unsynchronized Audio-Visual Events

arXiv: https://arxiv.org/abs/1804.07345 · HTML: https://mlsanity.com/paper/1804.07345/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Apr 2018.

## Built on by (most cited first, 4 of 4)

- [Real-Time Idling Vehicles Detection using Combined Audio-Visual Deep Learning](https://mlsanity.com/paper/2305.14579/) (2023)
- [Audiovisual transfer learning for audio tagging and sound event detection](https://mlsanity.com/paper/2106.05408/) (2021)
- [Where and When: Space-Time Attention for Audio-Visual Explanations](https://mlsanity.com/paper/2105.01517/) (2021)
- [Multi-Modulation Network for Audio-Visual Event Localization](https://mlsanity.com/paper/2108.11773/) (2021)

## Abstract (arXiv)

Audio-visual representation learning is an important task from the perspective of designing machines with the ability to understand complex events. To this end, we propose a novel multimodal framework that instantiates multiple instance learning. We show that the learnt representations are useful for classifying events and localizing their characteristic audio-visual elements. The system is trained using only video-level event labels without any timing information. An important feature of our method is its capacity to learn from unsynchronized audio-visual events. We achieve state-of-the-art results on a large-scale dataset of weakly-labeled audio event videos. Visualizations of localized visual regions and audio segments substantiate our system's efficacy, especially when dealing with noisy situations where modality-specific cues appear asynchronously.