# Saliency-based Multiple Region of Interest Detection from a Single 360deg image

arXiv: https://arxiv.org/abs/2209.03656 · HTML: https://mlsanity.com/paper/2209.03656/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 35 references that are in the map.
- First posted on arXiv 8 Sep 2022.

## Builds on (most cited first, 10 of 10)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Cube Padding for Weakly-Supervised Saliency Prediction in 360deg Videos](https://mlsanity.com/paper/1806.01320/) (2018)
- [SalGAN: Visual Saliency Prediction with Generative Adversarial Networks](https://mlsanity.com/paper/1701.01081/) (2017)
- [A Memory Network Approach for Story-based Temporal Summarization of 360deg Videos](https://mlsanity.com/paper/1805.02838/) (2018)
- [Pano2Vid: Automatic Cinematography for Watching 360circ Videos](https://mlsanity.com/paper/1612.02335/) (2016)
- [Deep 360 Pilot: Learning a Deep Agent for Piloting through 360deg Sports Video](https://mlsanity.com/paper/1705.01759/) (2017)
- [SalNet360: Saliency Maps for omni-directional images with CNN](https://mlsanity.com/paper/1709.06505/) (2017)
- [Making 360circ Video Watchable in 2D: Learning Videography for Click Free Viewing](https://mlsanity.com/paper/1703.00495/) (2017)

## Abstract (arXiv)

360deg images are informative -- it contains omnidirectional visual information around the camera. However, the areas that cover a 360deg image is much larger than the human's field of view, therefore important information in different view directions is easily overlooked. To tackle this issue, we propose a method for predicting the optimal set of Region of Interest (RoI) from a single 360deg image using the visual saliency as a clue. To deal with the scarce, strongly biased training data of existing single 360deg image saliency prediction dataset, we also propose a data augmentation method based on the spherical random data rotation. From the predicted saliency map and redundant candidate regions, we obtain the optimal set of RoIs considering both the saliency within a region and the Interaction-Over-Union (IoU) between regions. We conduct the subjective evaluation to show that the proposed method can select regions that properly summarize the input 360deg image.