# Joint Audio-Visual Idling Vehicle Detection with Streamlined Input Dependencies

arXiv: https://arxiv.org/abs/2410.21170 · HTML: https://mlsanity.com/paper/2410.21170/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 28 Oct 2024.

## Builds on (most cited first, 21 of 21)

- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [Audio-Visual Segmentation](https://mlsanity.com/paper/2207.05042/) (2022)
- [AVA-ActiveSpeaker: An Audio-Visual Dataset for Active Speaker Detection](https://mlsanity.com/paper/1901.01342/) (2019)
- [Vision Transformers are Parameter-Efficient Audio-Visual Learners](https://mlsanity.com/paper/2212.07983/) (2022)
- [Self-supervised Moving Vehicle Tracking with Stereo Sound](https://mlsanity.com/paper/1910.11760/) (2019)
- [You Only Watch Once: A Unified CNN Architecture for Real-Time Spatiotemporal Action Localization](https://mlsanity.com/paper/1911.06644/) (2019)
- [Resource Efficient 3D Convolutional Neural Networks](https://mlsanity.com/paper/1904.02422/) (2019)
- [Dense-Localizing Audio-Visual Events in Untrimmed Videos: A Large-Scale Benchmark and Baseline](https://mlsanity.com/paper/2303.12930/) (2023)
- [There is More than Meets the Eye: Self-Supervised Multi-Object Detection and Tracking with Sound by Distilling Multimodal Knowledge](https://mlsanity.com/paper/2103.01353/) (2021)
- [A Light Weight Model for Active Speaker Detection](https://mlsanity.com/paper/2303.04439/) (2023)
- [Learning Long-Term Spatial-Temporal Graphs for Active Speaker Detection](https://mlsanity.com/paper/2207.07783/) (2022)
- [Self-supervised object detection from audio-visual correspondence](https://mlsanity.com/paper/2104.06401/) (2021)
- [End-to-End Active Speaker Detection](https://mlsanity.com/paper/2203.14250/) (2022)
- [Optimizing Automated Picking Systems in Warehouse Robots Using Machine Learning](https://mlsanity.com/paper/2408.16633/) (2024)
- [Real-Time Idling Vehicles Detection using Combined Audio-Visual Deep Learning](https://mlsanity.com/paper/2305.14579/) (2023)
- [Deep Reinforcement Learning-based Obstacle Avoidance for Robot Movement in Warehouse Environments](https://mlsanity.com/paper/2409.14972/) (2024)
- [TalkNCE: Improving Active Speaker Detection with Talk-Aware Contrastive Learning](https://mlsanity.com/paper/2309.12306/) (2023)
- [Self-Supervised Moving Vehicle Detection from Audio-Visual Cues](https://mlsanity.com/paper/2201.12771/) (2022)
- [Remote Detection of Idling Cars Using Infrared Imaging and Deep Networks](https://mlsanity.com/paper/1804.10805/) (2018)
- [UniCon+: ICTCAS-UCAS Submission to the AVA-ActiveSpeaker Task at ActivityNet Challenge 2022](https://mlsanity.com/paper/2206.10861/) (2022)

## Built on by (most cited first, 14 of 14)

- [OFFSET: Segmentation-based Focus Shift Revision for Composed Image Retrieval](https://mlsanity.com/paper/2507.05631/) (2025)
- [Unmasking Digital Falsehoods: A Comparative Analysis of LLM-Based Misinformation Detection Strategies](https://mlsanity.com/paper/2503.00724/) (2025)
- [Disentanglement Analysis in Deep Latent Variable Models Matching Aggregate Posterior Distributions](https://mlsanity.com/paper/2501.15705/) (2025)
- [HAVT-IVD: Heterogeneity-Aware Cross-Modal Network for Audio-Visual Surveillance: Idling Vehicles Detection With Multichannel Audio and Multiscale Visual Cues](https://mlsanity.com/paper/2504.16102/) (2025)
- [First-place Solution for Streetscape Shop Sign Recognition Competition](https://mlsanity.com/paper/2501.02811/) (2025)
- [LPCAN: Lightweight Pyramid Cross-Attention Network for Rail Surface Defect Detection Using RGB-D Data](https://mlsanity.com/paper/2601.09118/) (2026)
- [Knowledge-Embedded and Hypernetwork-Guided Few-Shot Substation Meter Defect Image Generation Method](https://mlsanity.com/paper/2601.09238/) (2026)
- [MA-LipNet: Multi-Dimensional Attention Networks for Robust Lipreading](https://mlsanity.com/paper/2601.20881/) (2026)
- [The Diffusion Duet: Harmonizing Dual Channels with Wavelet Suppression for Image Separation](https://mlsanity.com/paper/2602.13361/) (2026)
- [Detecting Deepfakes with Multivariate Soft Blending and CLIP-based Image-Text Alignment](https://mlsanity.com/paper/2602.15903/) (2026)
- [Efficient Image Super-Resolution with Multi-Scale Spatial Adaptive Attention Networks](https://mlsanity.com/paper/2603.00118/) (2026)
- [A Deformable Attention-Based Detection Transformer with Cross-Scale Feature Fusion for Industrial Coil Spring Inspection](https://mlsanity.com/paper/2603.13429/) (2026)
- [JND-Guided Neural Watermarking with Spatial Transformer Decoding for Screen-Capture Robustness](https://mlsanity.com/paper/2603.26766/) (2026)
- [Towards Accurate and Robust Surveillance Roadside IVD via Trackletized Audio-Visual Reasoning](https://mlsanity.com/paper/2606.22299/) (2026)

## Abstract (arXiv)

Idling vehicle detection (IVD) can be helpful in monitoring and reducing unnecessary idling and can be integrated into real-time systems to address the resulting pollution and harmful products. The previous approach [13], a non-end-to-end model, requires extra user clicks to specify a part of the input, making system deployment more error-prone or even not feasible. In contrast, we introduce an end-to-end joint audio-visual IVD task designed to detect vehicles visually under three states: moving, idling and engine off. Unlike feature co-occurrence task such as audio-visual vehicle tracking, our IVD task addresses complementary features, where labels cannot be determined by a single modality alone. To this end, we propose AVIVD-Net, a novel network that integrates audio and visual features through a bidirectional attention mechanism. AVIVD-Net streamlines the input process by learning a joint feature space, reducing the deployment complexity of previous methods. Additionally, we introduce the AVIVD dataset, which is seven times larger than previous datasets, offering significantly more annotated samples to study the IVD problem. Our model achieves performance comparable to prior approaches, making it suitable for automated deployment. Furthermore, by evaluating AVIVDNet on the feature co-occurrence public dataset MAVD [23], we demonstrate its potential for extension to self-driving vehicle video-camera setups.