# A Deep Ranking Model for Spatio-Temporal Highlight Detection from a 360 Video

arXiv: https://arxiv.org/abs/1801.10312 · HTML: https://mlsanity.com/paper/1801.10312/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jan 2018.

## Built on by (most cited first, 16 of 16)

- [Pano-AVQA: Grounded Audio-Visual Question Answering on 360^circ Videos](https://mlsanity.com/paper/2110.05122/) (2021)
- [Deep Learning for Omnidirectional Vision: A Survey and New Perspectives](https://mlsanity.com/paper/2205.10468/) (2022)
- [Panoramic Vision Transformer for Saliency Detection in 360deg Videos](https://mlsanity.com/paper/2209.08956/) (2022)
- [LD-DETR: Loop Decoder DEtection TRansformer for Video Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2501.10787/) (2025)
- [Unsupervised Modality-Transferable Video Highlight Detection with Representation Activation Sequence Learning](https://mlsanity.com/paper/2403.09401/) (2024)
- [PR-Net: Preference Reasoning for Personalized Video Highlight Detection](https://mlsanity.com/paper/2109.01799/) (2021)
- [Unsupervised Video Highlight Detection by Learning from Audio and Visual Recurrence](https://mlsanity.com/paper/2407.13933/) (2024)
- [HighlightMe: Detecting Highlights from Human-Centric Videos](https://mlsanity.com/paper/2110.01774/) (2021)
- [Field-of-View IoU for Object Detection in 360deg Images](https://mlsanity.com/paper/2202.03176/) (2022)
- [Show Me What I Like: Detecting User-Specific Video Highlights Using Content-Based Multi-Head Attention](https://mlsanity.com/paper/2207.08352/) (2022)
- [An Integrated System for Spatio-Temporal Summarization of 360-degrees Videos](https://mlsanity.com/paper/2312.02576/) (2023)
- [Test-Time Adaptation for Video Highlight Detection Using Meta-Auxiliary Learning and Cross-Modality Hallucinations](https://mlsanity.com/paper/2508.04924/) (2025)
- [WinDB: HMD-free and Distortion-free Panoptic Video Fixation Learning](https://mlsanity.com/paper/2305.13901/) (2023)
- [CoSTL: Comprehensive Spatial-Temporal Representation Learning for Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2606.01149/) (2026)
- [A Human-Annotated Video Dataset for Training and Evaluation of 360-Degree Video Summarization Methods](https://mlsanity.com/paper/2406.02991/) (2024)
- [TSalV360: A Method and Dataset for Text-driven Saliency Detection in 360-Degrees Videos](https://mlsanity.com/paper/2509.26208/) (2025)

## Abstract (arXiv)

We address the problem of highlight detection from a 360 degree video by summarizing it both spatially and temporally. Given a long 360 degree video, we spatially select pleasantly-looking normal field-of-view (NFOV) segments from unlimited field of views (FOV) of the 360 degree video, and temporally summarize it into a concise and informative highlight as a selected subset of subshots. We propose a novel deep ranking model named as Composition View Score (CVS) model, which produces a spherical score map of composition per video segment, and determines which view is suitable for highlight via a sliding window kernel at inference. To evaluate the proposed framework, we perform experiments on the Pano2Vid benchmark dataset and our newly collected 360 degree video highlight dataset from YouTube and Vimeo. Through evaluation using both quantitative summarization metrics and user studies via Amazon Mechanical Turk, we demonstrate that our approach outperforms several state-of-the-art highlight detection methods. We also show that our model is 16 times faster at inference than AutoCam, which is one of the first summarization algorithms of 360 degree videos