# A Spatial-Temporal Dual-Mode Mixed Flow Network for Panoramic Video Salient Object Detection

arXiv: https://arxiv.org/abs/2310.09016 · HTML: https://mlsanity.com/paper/2310.09016/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 29 references that are in the map.
- First posted on arXiv 13 Oct 2023.

## Builds on (most cited first, 20 of 20)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Salient Object Detection in the Deep Learning Era: An In-Depth Survey](https://mlsanity.com/paper/1904.09146/) (2019)
- [Motion Guided Attention for Video Salient Object Detection](https://mlsanity.com/paper/1909.07061/) (2019)
- [A Unified Transformer Framework for Group-based Segmentation: Co-Segmentation, Co-Saliency Detection and Video Salient Object Detection](https://mlsanity.com/paper/2203.04708/) (2022)
- [Video Salient Object Detection via Fully Convolutional Networks](https://mlsanity.com/paper/1702.00871/) (2017)
- [Depth Quality-Inspired Feature Manipulation for Efficient RGB-D Salient Object Detection](https://mlsanity.com/paper/2107.01779/) (2021)
- [Exploring Rich and Efficient Spatial Temporal Interactions for Real Time Video Salient Object Detection](https://mlsanity.com/paper/2008.02973/) (2020)
- [SALISA: Saliency-based Input Sampling for Efficient Video Object Detection](https://mlsanity.com/paper/2204.02397/) (2022)
- [Motion-aware Memory Network for Fast Video Salient Object Detection](https://mlsanity.com/paper/2208.00946/) (2022)
- [Depth-Cooperated Trimodal Network for Video Salient Object Detection](https://mlsanity.com/paper/2202.06060/) (2022)
- [Panoramic Video Salient Object Detection with Ambisonic Audio Guidance](https://mlsanity.com/paper/2211.14419/) (2022)
- [A Novel Video Salient Object Detection Method via Semi-supervised Motion Quality Perception](https://mlsanity.com/paper/2008.02966/) (2020)
- [DS-Net: Dynamic Spatiotemporal Network for Video Salient Object Detection](https://mlsanity.com/paper/2012.04886/) (2020)
- [Guidance and Teaching Network for Video Salient Object Detection](https://mlsanity.com/paper/2105.10110/) (2021)
- [SHD360: A Benchmark Dataset for Salient Human Detection in 360deg Videos](https://mlsanity.com/paper/2105.11578/) (2021)
- [A Novel Long-term Iterative Mining Scheme for Video Salient Object Detection](https://mlsanity.com/paper/2206.09564/) (2022)
- [Video Salient Object Detection via Adaptive Local-Global Refinement](https://mlsanity.com/paper/2104.14360/) (2021)
- [ASOD60K: An Audio-Induced Salient Object Detection Dataset for Panoramic Videos](https://mlsanity.com/paper/2107.11629/) (2021)
- [Salient Object Detection in Video using Deep Non-Local Neural Networks](https://mlsanity.com/paper/1810.07097/) (2018)

## Abstract (arXiv)

Salient object detection (SOD) in panoramic video is still in the initial exploration stage. The indirect application of 2D video SOD method to the detection of salient objects in panoramic video has many unmet challenges, such as low detection accuracy, high model complexity, and poor generalization performance. To overcome these hurdles, we design an Inter-Layer Attention (ILA) module, an Inter-Layer weight (ILW) module, and a Bi-Modal Attention (BMA) module. Based on these modules, we propose a Spatial-Temporal Dual-Mode Mixed Flow Network (STDMMF-Net) that exploits the spatial flow of panoramic video and the corresponding optical flow for SOD. First, the ILA module calculates the attention between adjacent level features of consecutive frames of panoramic video to improve the accuracy of extracting salient object features from the spatial flow. Then, the ILW module quantifies the salient object information contained in the features of each level to improve the fusion efficiency of the features of each level in the mixed flow. Finally, the BMA module improves the detection accuracy of STDMMF-Net. A large number of subjective and objective experimental results testify that the proposed method demonstrates better detection accuracy than the state-of-the-art (SOTA) methods. Moreover, the comprehensive performance of the proposed method is better in terms of memory required for model inference, testing time, complexity, and generalization performance.