ASOD60K: An Audio-Induced Salient Object Detection Dataset for Panoramic Videos
Not on the map. Why?
Abstract (arXiv)
Exploring to what humans pay attention in dynamic panoramic scenes is useful for many fundamental applications, including augmented reality (AR) in retail, AR-powered recruitment, and visual language navigation. With this goal in mind, we propose PV-SOD, a new task that aims to segment salient objects from panoramic videos. In contrast to existing fixation-/object-level saliency detection tasks, we focus on audio-induced salient object detection (SOD), where the salient objects are labeled with the guidance of audio-induced eye movements. To support this task, we collect the first large-scale dataset, named ASOD60K, which contains 4K-resolution video frames annotated with a six-level hierarchy, thus distinguishing itself with richness, diversity and quality. Specifically, each sequence is marked with both its super-/sub-class, with objects of each sub-class being further annotated with human eye fixations, bounding boxes, object-/instance-level masks, and associated attributes (e.g., geometrical distortion). These coarse-to-fine annotations enable detailed analysis for PV-SOD modelling, e.g., determining the major challenges for existing SOD models, and predicting scanpaths to study the long-term eye fixation behaviors of humans. We systematically benchmark 11 representative approaches on ASOD60K and derive several interesting findings. We hope this study could serve as a good starting point for advancing SOD research towards panoramic videos. The dataset and benchmark will be made publicly available at https://github.com/PanoAsh/ASOD60K.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 59 of its 107 references that are in the map. First posted on arXiv 24 Jul 2021.
Builds on
59 of 107 references.
References · 59
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2017Structure-measure: A New Way to Evaluate Foreground Mapscited by 480
- 2018Spherical CNNscited by 452
- 2018Enhanced-alignment Measure for Binary Foreground Map Evaluationcited by 370
- 2019Gauge Equivariant Convolutional Networks and the Icosahedral CNNcited by 262
- 2017Learning SO(3) Equivariant Representations with Spherical CNNscited by 244
- 2019EGNet:Edge Guidance Network for Salient Object Detectioncited by 237
- 2019Cascaded Partial Decoder for Fast and Accurate Salient Object Detectioncited by 216
- 2020Multi-scale Interactive Network for Salient Object Detectioncited by 212
- 2019A Simple Pooling-Based Design for Real-Time Salient Object Detectioncited by 211
- 2014The Secrets of Salient Object Segmentationcited by 200
38 more
- 2015Visual Saliency Based on Multiscale Deep Featurescited by 185
- 2019Salient Object Detection in the Deep Learning Era: An In-Depth Surveycited by 163
- 2020See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networkscited by 138
- 2019Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarkscited by 133
- 2021Salient Object Detection via Integrity Learningcited by 133
- 2015Salient Object Detection: A Benchmarkcited by 130
- 2020Suppress and Balance: A Simple Gated Network for Salient Object Detectioncited by 126
- 2020Label Decoupling Framework for Salient Object Detectioncited by 111
- 2020Global Context-Aware Progressive Aggregation Network for Salient Object Detectioncited by 109
- 2014Salient Object Detection: A Surveycited by 98
- 2018Self-Supervised Generation of Spatial Audio for 360 Videocited by 93
- 2019Towards High-Resolution Salient Object Detectioncited by 92
- 2019Tangent Images for Mitigating Spherical Distortioncited by 79
- 2020Highly Efficient Salient Object Detection with 100K Parameterscited by 79
- 2020JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detectioncited by 79
- 2019Spherical CNNs on Unstructured Gridscited by 76
- 2020RGB-D Salient Object Detection: A Surveycited by 73
- 2019Motion Guided Attention for Video Salient Object Detectioncited by 72
- 2020Siamese Network for RGB-D Salient Object Detection and Beyondcited by 67
- 2018Revisiting Video Saliency: A Large-scale Benchmark and a New Modelcited by 66
- 2018Cube Padding for Weakly-Supervised Saliency Prediction in 360deg Videoscited by 66
- 2020Dense Attention Fluid Network for Salient Object Detection in Optical Remote Sensing Imagescited by 61
- 2019RGB-T Image Saliency Detection via Collaborative Graph Learningcited by 58
- 2018Kernel Transformer Networks for Compact Spherical Convolutioncited by 56
- 2019Semi-Supervised Video Salient Object Detection Using Pseudo-Labelscited by 54
- 2020Re-thinking Co-Salient Object Detectioncited by 54
- 2019Orientation-aware Semantic Segmentation on Icosahedron Spherescited by 52
- 2018Salient Objects in Clutter: Bringing Salient Object Detection to the Foregroundcited by 47
- 2021Boundary-Aware Segmentation Network for Mobile and Web Applicationscited by 41
- 2020STAViS: Spatio-Temporal AudioVisual Saliency Networkcited by 40
- 2017Instance-Level Salient Object Segmentationcited by 38
- 2020Learning Selective Mutual Attention and Contrast for RGB-D Saliency Detectioncited by 38
- 2017Predicting Head Movement in Panoramic Video: A Deep Reinforcement Learning Approachcited by 34
- 2021Group Collaborative Learning for Co-Salient Object Detectioncited by 32
- 2020Semantic Object Prediction and Spatial Sound Super-Resolution with Binaural Soundscited by 26
- 2019360-Indoor: Towards Learning Real-World Objects in 360deg Indoor Equirectangular Imagescited by 25
- 2016Pano2Vid: Automatic Cinematography for Watching 360circ Videoscited by 22
- 2017Deep 360 Pilot: Learning a Deep Agent for Piloting through 360deg Sports Videocited by 22
50 most cited of 59 All 59 (JSON)
Built on by
Citing papers per year: 2022: 2; 2023: 1.
3 papers.