# Learnable Optimal Sequential Grouping for Video Scene Detection

arXiv: https://arxiv.org/abs/2205.08249 · HTML: https://mlsanity.com/paper/2205.08249/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 40 references that are in the map.
- First posted on arXiv 17 May 2022.

## Builds on (most cited first, 9 of 9)

- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [Motion-Appearance Co-Memory Networks for Video Question Answering](https://mlsanity.com/paper/1803.10906/) (2018)
- [The State of the Art in Integrating Machine Learning into Visual Analytics](https://mlsanity.com/paper/1802.07954/) (2018)
- [A Deep Siamese Network for Scene Detection in Broadcast Videos](https://mlsanity.com/paper/1510.08893/) (2015)
- [Combining Learned and Analytical Models for Predicting Action Effects from Sensory Data](https://mlsanity.com/paper/1710.04102/) (2017)
- [ILS-SUMM: Iterated Local Search for Unsupervised Video Summarization](https://mlsanity.com/paper/1912.03650/) (2019)

## Built on by (most cited first, 3 of 3)

- [OS-MSL: One Stage Multimodal Sequential Link Framework for Scene Segmentation and Classification](https://mlsanity.com/paper/2207.01241/) (2022)
- [Multimodal Chaptering for Long-Form TV Newscast Video](https://mlsanity.com/paper/2406.17590/) (2024)
- [GHQ: Grouped Hybrid Q Learning for Heterogeneous Cooperative Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2303.01070/) (2023)

## Abstract (arXiv)

Video scene detection is the task of dividing videos into temporal semantic chapters. This is an important preliminary step before attempting to analyze heterogeneous video content. Recently, Optimal Sequential Grouping (OSG) was proposed as a powerful unsupervised solution to solve a formulation of the video scene detection problem. In this work, we extend the capabilities of OSG to the learning regime. By giving the capability to both learn from examples and leverage a robust optimization formulation, we can boost performance and enhance the versatility of the technology. We present a comprehensive analysis of incorporating OSG into deep learning neural networks under various configurations. These configurations include learning an embedding in a straight-forward manner, a tailored loss designed to guide the solution of OSG, and an integrated model where the learning is performed through the OSG pipeline. With thorough evaluation and analysis, we assess the benefits and behavior of the various configurations, and show that our learnable OSG approach exhibits desirable behavior and enhanced performance compared to the state of the art.