# Identify, locate and separate: Audio-visual object extraction in large video collections using weak supervision

arXiv: https://arxiv.org/abs/1811.04000 · HTML: https://mlsanity.com/paper/1811.04000/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Nov 2018.

## Built on by (most cited first, 3 of 3)

- [Listen to Interpret: Post-hoc Interpretability for Audio Networks with NMF](https://mlsanity.com/paper/2202.11479/) (2022)
- [Tackling Interpretability in Audio Classification Networks with Non-negative Matrix Factorization](https://mlsanity.com/paper/2305.07132/) (2023)
- [Induction Network: Audio-Visual Modality Gap-Bridging for Self-Supervised Sound Source Localization](https://mlsanity.com/paper/2308.04767/) (2023)

## Abstract (arXiv)

We tackle the problem of audiovisual scene analysis for weakly-labeled data. To this end, we build upon our previous audiovisual representation learning framework to perform object classification in noisy acoustic environments and integrate audio source enhancement capability. This is made possible by a novel use of non-negative matrix factorization for the audio modality. Our approach is founded on the multiple instance learning paradigm. Its effectiveness is established through experiments over a challenging dataset of music instrument performance videos. We also show encouraging visual object localization results.