mlsanityData to 7 Oct 2026

Detection of Audio-Video Synchronization Errors Via Event Detection

arXiv 2104.10116 ↗First posted 20 Apr 2021cs.MM cs.CV cs.SDcited by 6

Not on the map. Why?

Abstract (arXiv)

We present a new method and a large-scale database to detect audio-video synchronization(A/V sync) errors in tennis videos. A deep network is trained to detect the visual signature of the tennis ball being hit by the racquet in the video stream. Another deep network is trained to detect the auditory signature of the same event in the audio stream. During evaluation, the audio stream is searched by the audio network for the audio event of the ball being hit. If the event is found in audio, the neighboring interval in video is searched for the corresponding visual signature. If the event is not found in the video stream but is found in the audio stream, A/V sync error is flagged. We developed a large-scaled database of 504,300 frames from 6 hours of videos of tennis events, simulated A/V sync errors, and found our method achieves high accuracy on the task.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 13 of its 22 references that are in the map. First posted on arXiv 20 Apr 2021.

Builds on

13 of 22 references.

References · 13

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  3. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  4. 2018Deep Audio-Visual Speech Recognitioncited by 318
  5. 2018Cooperative Learning of Audio and Video Models from Self-Supervised Synchronizationcited by 246
  6. 2019EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognitioncited by 226
  7. 2019Deep Learning for Audio Signal Processingcited by 92
  8. 2020Rethinking CNN Models for Audio Classificationcited by 48
  9. 2018Automatic acoustic detection of birds through deep learning: the first Bird Audio Detection challengecited by 44
  10. 2020ESResNet: Environmental Sound Classification Based on Visual Domain Modelscited by 31
  11. 2018On Attention Modules for Audio-Visual Synchronizationcited by 13
  12. 2019Urban Sound Tagging using Convolutional Neural Networkscited by 7
1 more
  1. 2017Convolutional Recurrent Neural Networks for Bird Audio Detectioncited by 5

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 0; 2024: 1; 2025: 2.

6 papers.

  1. 2021Audio-Visual Synchronisation in the wildcited by 39
  2. 2022Audio-visual Generalised Zero-shot Learning with Cross-modal Attention and Languagecited by 27
  3. 2022Sparse in Space and Time: Audio-visual Synchronisation with Trainable Selectorscited by 25
  4. 2025VGGSounder: Audio-Visual Evaluations for Foundation Modelscited by 8
  5. 2024Efficient Multiscale Multimodal Bottleneck Transformer for Audio-Video Classificationcited by 4
  6. 2025Scaling Up Audio-Synchronized Visual Animation: An Efficient Training Paradigmcited by 2