A Curated Dataset of Urban Scenes for Audio-Visual Scene Analysis
Not on the map. Why?
Abstract (arXiv)
This paper introduces a curated dataset of urban scenes for audio-visual scene analysis which consists of carefully selected and recorded material. The data was recorded in multiple European cities, using the same equipment, in multiple locations for each scene, and is openly available. We also present a case study for audio-visual scene recognition and show that joint modeling of audio and visual modalities brings significant performance gain compared to state of the art uni-modal systems. Our approach obtained an 84.8% accuracy compared to 75.8% for the audio-only and 68.4% for the video-only equivalent systems.
Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Oct 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 2; 2022: 5; 2023: 2; 2024: 4; 2025: 2; 2026: 1.
16 papers.
Sort:
By area:
- 2021Wav2CLIP: Learning Robust Audio Representations From CLIPcited by 210
- 2022SpecGrad: Diffusion Probabilistic Model based Neural Vocoder with Adaptive Noise Spectral Shapingcited by 37
- 2025InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialoguecited by 27
- 2023Miipher: A Robust Speech Restoration Model Integrating Self-Supervised Speech and Text Representationscited by 22
- 2025DiTSE: High-Fidelity Generative Speech Enhancement via Latent Diffusion Transformerscited by 6
- 2022Attentional Graph Convolutional Network for Structure-aware Audio-Visual Scene Classificationcited by 5
- 2023Versatile audio-visual learning for emotion recognitioncited by 5
- 2024Audio Enhancement for Computer Audition -- An Iterative Training Paradigm Using Sample Importancecited by 2
- 2021Squeeze-Excitation Convolutional Recurrent Neural Networks for Audio-Visual Scene Classificationcited by 0
- 2022A study on joint modeling and data augmentation of multi-modalities for audio-visual scene classificationcited by 0
- 2022Audio-Visual Scene Classification Using A Transfer Learning Based Joint Optimization Strategycited by 0
- 2022Multi-Source Transformer Architectures for Audiovisual Scene Classificationcited by 0
4 more
- 2024Visual and audio scene classification for detecting discrepancies in video: a baseline method and experimental protocolcited by 0
- 2024A Toolchain for Comprehensive Audio/Video Analysis Using Deep Learning Based Multimodal Approach (A use case of riot or violent context detection)cited by 0
- 2024Bridging the Gap between Text, Audio, Image, and Any Sequence: A Novel Approach using Gloss-based Annotationcited by 0
- 2026AVSD-Scenes: A Dataset for Audio-Visual Description of Urban Scenescited by 0
- 2026AVSD-Scenes: A Dataset for Audio-Visual Description of Urban Scenes1 Oct 2026
- 2025InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialogue15 Oct 2025
- 2025DiTSE: High-Fidelity Generative Speech Enhancement via Latent Diffusion Transformers13 Apr 2025
- 2024Bridging the Gap between Text, Audio, Image, and Any Sequence: A Novel Approach using Gloss-based Annotation4 Oct 2024
- 2024Audio Enhancement for Computer Audition -- An Iterative Training Paradigm Using Sample Importance12 Aug 2024
- 2024A Toolchain for Comprehensive Audio/Video Analysis Using Deep Learning Based Multimodal Approach (A use case of riot or violent context detection)2 May 2024
- 2024Visual and audio scene classification for detecting discrepancies in video: a baseline method and experimental protocol1 May 2024
- 2023Versatile audio-visual learning for emotion recognition12 May 2023
- 2023Miipher: A Robust Speech Restoration Model Integrating Self-Supervised Speech and Text Representations3 Mar 2023
- 2022Attentional Graph Convolutional Network for Structure-aware Audio-Visual Scene Classification31 Dec 2022
- 2022Multi-Source Transformer Architectures for Audiovisual Scene Classification18 Oct 2022
- 2022Audio-Visual Scene Classification Using A Transfer Learning Based Joint Optimization Strategy25 Apr 2022
4 more
- 2022SpecGrad: Diffusion Probabilistic Model based Neural Vocoder with Adaptive Noise Spectral Shaping31 Mar 2022
- 2022A study on joint modeling and data augmentation of multi-modalities for audio-visual scene classification7 Mar 2022
- 2021Wav2CLIP: Learning Robust Audio Representations From CLIP21 Oct 2021
- 2021Squeeze-Excitation Convolutional Recurrent Neural Networks for Audio-Visual Scene Classification28 Jul 2021
- 2021Wav2CLIP: Learning Robust Audio Representations From CLIPcited by 210
- 2025InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialoguecited by 27
- 2023Miipher: A Robust Speech Restoration Model Integrating Self-Supervised Speech and Text Representationscited by 22
- 2022Attentional Graph Convolutional Network for Structure-aware Audio-Visual Scene Classificationcited by 5
- 2023Versatile audio-visual learning for emotion recognitioncited by 5
- 2024Audio Enhancement for Computer Audition -- An Iterative Training Paradigm Using Sample Importancecited by 2
- 2021Squeeze-Excitation Convolutional Recurrent Neural Networks for Audio-Visual Scene Classificationcited by 0
- 2022A study on joint modeling and data augmentation of multi-modalities for audio-visual scene classificationcited by 0
- 2022Audio-Visual Scene Classification Using A Transfer Learning Based Joint Optimization Strategycited by 0
- 2022Multi-Source Transformer Architectures for Audiovisual Scene Classificationcited by 0
- 2024Visual and audio scene classification for detecting discrepancies in video: a baseline method and experimental protocolcited by 0
- 2024A Toolchain for Comprehensive Audio/Video Analysis Using Deep Learning Based Multimodal Approach (A use case of riot or violent context detection)cited by 0
12 most cited of 14 All 14 (JSON)
- 2026AVSD-Scenes: A Dataset for Audio-Visual Description of Urban Scenes1 Oct 2026
- 2025InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialogue15 Oct 2025
- 2024Bridging the Gap between Text, Audio, Image, and Any Sequence: A Novel Approach using Gloss-based Annotation4 Oct 2024
- 2024Audio Enhancement for Computer Audition -- An Iterative Training Paradigm Using Sample Importance12 Aug 2024
- 2024A Toolchain for Comprehensive Audio/Video Analysis Using Deep Learning Based Multimodal Approach (A use case of riot or violent context detection)2 May 2024
- 2024Visual and audio scene classification for detecting discrepancies in video: a baseline method and experimental protocol1 May 2024
- 2023Versatile audio-visual learning for emotion recognition12 May 2023
- 2023Miipher: A Robust Speech Restoration Model Integrating Self-Supervised Speech and Text Representations3 Mar 2023
- 2022Attentional Graph Convolutional Network for Structure-aware Audio-Visual Scene Classification31 Dec 2022
- 2022Multi-Source Transformer Architectures for Audiovisual Scene Classification18 Oct 2022
- 2022Audio-Visual Scene Classification Using A Transfer Learning Based Joint Optimization Strategy25 Apr 2022
- 2022A study on joint modeling and data augmentation of multi-modalities for audio-visual scene classification7 Mar 2022
12 newest of 14 All 14 (JSON)