mlsanityData to 13 Oct 2026

Panoramic Video Salient Object Detection with Ambisonic Audio Guidance

arXiv 2211.14419 ↗First posted 26 Nov 2022cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

Video salient object detection (VSOD), as a fundamental computer vision problem, has been extensively discussed in the last decade. However, all existing works focus on addressing the VSOD problem in 2D scenarios. With the rapid development of VR devices, panoramic videos have been a promising alternative to 2D videos to provide immersive feelings of the real world. In this paper, we aim to tackle the video salient object detection problem for panoramic videos, with their corresponding ambisonic audios. A multimodal fusion module equipped with two pseudo-siamese audio-visual context fusion (ACF) blocks is proposed to effectively conduct audio-visual interaction. The ACF block equipped with spherical positional encoding enables the fusion in the 3D context to capture the spatial correspondence between pixels and sound sources from the equirectangular frames and ambisonic audios. Experimental results verify the effectiveness of our proposed components and demonstrate that our method achieves state-of-the-art performance on the ASOD60K dataset.

Key facts. Cited by 9 papers in the map (data through 13 Oct 2026). Builds on 45 of its 70 references that are in the map. First posted on arXiv 26 Nov 2022.

Builds on

45 of 70 references.

References · 45

  1. 2017Attention Is All You Needcited by 60,106
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
  4. 2016Feature Pyramid Networks for Object Detectioncited by 5,027
  5. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,287
  6. 2017Structure-measure: A New Way to Evaluate Foreground Mapscited by 480
  7. 2018Enhanced-alignment Measure for Binary Foreground Map Evaluationcited by 370
  8. 2019Cross-Modal Self-Attention Network for Referring Image Segmentationcited by 263
  9. 2019Cascaded Partial Decoder for Fast and Accurate Salient Object Detectioncited by 216
  10. 2020Multi-scale Interactive Network for Salient Object Detectioncited by 212
  11. 2022Language as Queries for Referring Video Object Segmentationcited by 180
  12. 2019F3Net: Fusion, Feedback and Focus for Salient Object Detectioncited by 176
33 more
  1. 2022Audio-Visual Segmentationcited by 149
  2. 2020See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networkscited by 138
  3. 2015Salient Object Detection: A Benchmarkcited by 130
  4. 2020Suppress and Balance: A Simple Gated Network for Salient Object Detectioncited by 126
  5. 2020Label Decoupling Framework for Salient Object Detectioncited by 111
  6. 2018Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networkscited by 84
  7. 2020Highly Efficient Salient Object Detection with 100K Parameterscited by 79
  8. 2019Motion Guided Attention for Video Salient Object Detectioncited by 72
  9. 2021SwiftNet: Real-time Video Object Segmentationcited by 67
  10. 2018Cube Padding for Weakly-Supervised Saliency Prediction in 360deg Videoscited by 66
  11. 2020Motion-Attentive Transition for Zero-Shot Video Object Segmentationcited by 62
  12. 2022SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detectioncited by 59
  13. 2020Making a Case for 3D Convolutions for Object Segmentation in Videoscited by 56
  14. 2019Semi-Supervised Video Salient Object Detection Using Pseudo-Labelscited by 54
  15. 2022A Unified Transformer Framework for Group-based Segmentation: Co-Segmentation, Co-Saliency Detection and Video Salient Object Detectioncited by 51
  16. 2017Video Salient Object Detection via Fully Convolutional Networkscited by 46
  17. 2020STAViS: Spatio-Temporal AudioVisual Saliency Networkcited by 40
  18. 2021LGD: Label-guided Self-distillation for Object Detectioncited by 19
  19. 2021Video Salient Object Detection via Contrastive Features and Attention Modulescited by 13
  20. 2021Hybrid Instance-aware Temporal Fusion for Online Video Instance Segmentationcited by 13
  21. 2022Semantic-aligned Fusion Transformer for One-shot Object Detectioncited by 13
  22. 2021Video Instance Segmentation by Instance Flow Assemblycited by 12
  23. 2021Unifying Global-Local Representations in Salient Object Detection with Transformercited by 11
  24. 2022Alignment-guided Temporal Attention for Video Action Recognitioncited by 10
  25. 2016Unsupervised Learning of Important Objects from First-Person Videoscited by 9
  26. 2021L3DAS21 Challenge: Machine Learning for 3D Audio Signal Processingcited by 9
  27. 2021Towards Cross-Modal Forgery Detection and Localization on Live Surveillance Videoscited by 8
  28. 2019A Self Validation Network for Object-Level Human Attention Estimationcited by 5
  29. 2022Online Video Instance Segmentation via Robust Context Fusioncited by 5
  30. 2018Saliency Map Estimation for Omni-Directional Image Considering Prior Distributionscited by 4
  31. 2022Forgery Attack Detection in Surveillance Video Streams Using Wi-Fi Channel State Informationcited by 4
  32. 2022Self-supervised Multi-Modal Video Forgery Attack Detectioncited by 4
  33. 2021ASOD60K: An Audio-Induced Salient Object Detection Dataset for Panoramic Videoscited by 3

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 1; 2023: 5; 2024: 1; 2025: 1; 2026: 1.

9 papers.

  1. 2023QDFormer: Towards Robust Audiovisual Segmentation in Complex Environments with Quantization-based Semantic Decompositioncited by 19
  2. 2023PanoVOS: Bridging Non-panoramic and Panoramic Views with Transformer for Video Segmentationcited by 18
  3. 2025EDM: Equirectangular Projection-Oriented Dense Kernelized Feature Matchingcited by 7
  4. 2024RTGen: Generating Region-Text Pairs for Open-Vocabulary Object Detectioncited by 4
  5. 2023Completing Visual Objects via Bridging Generation and Segmentationcited by 2
  6. 2026Panoramic Scene Understanding: A Survey from Distortion-Aware Engineering to Sphere-Native Modelingcited by 1
  7. 2022Recovering Surveillance Video Using RF Cuescited by 0
  8. 2023Object Segmentation with Audio Contextcited by 0
  9. 2023A Spatial-Temporal Dual-Mode Mixed Flow Network for Panoramic Video Salient Object Detectioncited by 0