# AutoShot: A Short Video Dataset and State-of-the-Art Shot Boundary Detection

arXiv: https://arxiv.org/abs/2304.06116 · HTML: https://mlsanity.com/paper/2304.06116/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 46 references that are in the map.
- First posted on arXiv 12 Apr 2023.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification](https://mlsanity.com/paper/1712.04851/) (2017)
- [Single Path One-Shot Neural Architecture Search with Uniform Sampling](https://mlsanity.com/paper/1904.00420/) (2019)
- [Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks](https://mlsanity.com/paper/1711.10305/) (2017)
- [COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysis](https://mlsanity.com/paper/1903.02874/) (2019)
- [TransNet V2: An effective deep network architecture for fast shot transition detection](https://mlsanity.com/paper/2008.04838/) (2020)
- [Generic Event Boundary Detection: A Benchmark for Event Segmentation](https://mlsanity.com/paper/2101.10511/) (2021)
- [Co-occurrence Feature Learning for Skeleton based Action Recognition using Regularized Deep LSTM Networks](https://mlsanity.com/paper/1603.07772/) (2016)
- [A Local-to-Global Approach to Multi-modal Movie Scene Segmentation](https://mlsanity.com/paper/2004.02678/) (2020)
- [AnatomyNet: Deep Learning for Fast and Fully Automated Whole-volume Segmentation of Head and Neck Anatomy](https://mlsanity.com/paper/1808.05238/) (2018)
- [OpenBox: A Generalized Black-box Optimization Service](https://mlsanity.com/paper/2106.00421/) (2021)
- [DeepLung: Deep 3D Dual Path Nets for Automated Pulmonary Nodule Detection and Classification](https://mlsanity.com/paper/1801.09555/) (2018)
- [Shot Contrastive Self-Supervised Learning for Scene Boundary Detection](https://mlsanity.com/paper/2104.13537/) (2021)
- [A Deep Siamese Network for Scene Detection in Broadcast Videos](https://mlsanity.com/paper/1510.08893/) (2015)
- [Shifted Chunk Transformer for Spatio-Temporal Representational Learning](https://mlsanity.com/paper/2108.11575/) (2021)
- [Fast Video Shot Transition Localization with Deep Structured Models](https://mlsanity.com/paper/1808.04234/) (2018)
- [Large-scale, Fast and Accurate Shot Boundary Detection through Spatio-temporal Convolutional Neural Networks](https://mlsanity.com/paper/1705.03281/) (2017)
- [Ridiculously Fast Shot Boundary Detection with Fully Convolutional Neural Networks](https://mlsanity.com/paper/1705.08214/) (2017)
- [Filter Grafting for Deep Neural Networks](https://mlsanity.com/paper/2001.05868/) (2020)
- [Detect Faces Efficiently: A Survey and Evaluations](https://mlsanity.com/paper/2112.01787/) (2021)
- [A Systematic IoU-Related Method: Beyond Simplified Regression for Better Localization](https://mlsanity.com/paper/2112.01793/) (2021)
- [Overview of Tencent Multi-modal Ads Video Understanding Challenge](https://mlsanity.com/paper/2109.07951/) (2021)
- [DeepEM: Deep 3D ConvNets With EM For Weakly Supervised Pulmonary Nodule Detection](https://mlsanity.com/paper/1805.05373/) (2018)
- [SpeechNAS: Towards Better Trade-off between Latency and Accuracy for Large-Scale Speaker Verification](https://mlsanity.com/paper/2109.08839/) (2021)

## Built on by (most cited first, 23 of 23)

- [InternVideo2: Scaling Foundation Models for Multimodal Video Understanding](https://mlsanity.com/paper/2403.15377/) (2024)
- [SafeWatch: An Efficient Safety-Policy Following Video Guardrail Model with Transparent Explanations](https://mlsanity.com/paper/2412.06878/) (2024)
- [LinVT: Empower Your Image-level Large Language Model to Understand Videos](https://mlsanity.com/paper/2412.05185/) (2024)
- [MoGA: Mixture-of-Groups Attention for End-to-End Long Video Generation](https://mlsanity.com/paper/2510.18692/) (2025)
- [VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Tree](https://mlsanity.com/paper/2510.22693/) (2025)
- [Generative AI for Film Creation: A Survey of Recent Advances](https://mlsanity.com/paper/2504.08296/) (2025)
- [YouTube SFV+HDR Quality Dataset](https://mlsanity.com/paper/2406.05305/) (2024)
- [A Benchmark and Multi-Agent System for Instruction-driven Cinematic Video Compilation](https://mlsanity.com/paper/2604.10456/) (2026)
- [OmniShotCut: Holistic Relational Shot Boundary Detection with Shot-Query Transformer](https://mlsanity.com/paper/2604.24762/) (2026)
- [An Empirical Comparison of Video Frame Sampling Methods for Multi-Modal RAG Retrieval](https://mlsanity.com/paper/2408.03340/) (2024)
- [MTVCraft: Tokenizing 4D Motion for Arbitrary Character Animation](https://mlsanity.com/paper/2505.10238/) (2025)
- [DUAL-VAD: Dual Benchmarks and Anomaly-Focused Sampling for Video Anomaly Detection](https://mlsanity.com/paper/2509.11605/) (2025)
- [Enhancing Video Large Language Models with Structured Multi-Video Collaborative Reasoning](https://mlsanity.com/paper/2509.13161/) (2025)
- [LibraGen: Playing a Balance Game in Subject-Driven Video Generation](https://mlsanity.com/paper/2603.13506/) (2026)
- [SmartDirector: Keyframe-Conditioned Cinematic Video Generation with Narrative Pacing Control](https://mlsanity.com/paper/2605.27891/) (2026)
- [Shot-Aware Frame Sampling for Video Understanding](https://mlsanity.com/paper/2603.17374/) (2026)
- [TransVLM: A Vision-Language Framework and Benchmark for Detecting Any Shot Transitions](https://mlsanity.com/paper/2604.27975/) (2026)
- [Faster than real-time detection of shot boundaries, sampling structure and dynamic keyframes in video](https://mlsanity.com/paper/2502.09202/) (2025)
- [ViDRiP-LLaVA: A Dataset and Benchmark for Diagnostic Reasoning from Pathology Videos](https://mlsanity.com/paper/2505.04192/) (2025)
- [U-CESE: Unified Clip-based Event Search Engine for AI Challenge HCMC 2025](https://mlsanity.com/paper/2605.23274/) (2026)
- [JenBridge: Adaptive Long-Form Video Soundtracking across Scene Transitions](https://mlsanity.com/paper/2606.01703/) (2026)
- [PERSIST: Persistent-State Discrimination for Shot Boundary Detection](https://mlsanity.com/paper/2608.29287/) (2026)
- [Moving Forward with Video Saliency: A New Dataset and Benchmark where Motion Matters](https://mlsanity.com/paper/2610.03276/) (2026)

## Abstract (arXiv)

The short-form videos have explosive popularity and have dominated the new social media trends. Prevailing short-video platforms,~textite.g., Kuaishou (Kwai), TikTok, Instagram Reels, and YouTube Shorts, have changed the way we consume and create content. For video content creation and understanding, the shot boundary detection (SBD) is one of the most essential components in various scenarios. In this work, we release a new public Short video sHot bOundary deTection dataset, named SHOT, consisting of 853 complete short videos and 11,606 shot annotations, with 2,716 high quality shot boundary annotations in 200 test videos. Leveraging this new data wealth, we propose to optimize the model design for video SBD, by conducting neural architecture search in a search space encapsulating various advanced 3D ConvNets and Transformers. Our proposed approach, named AutoShot, achieves higher F1 scores than previous state-of-the-art approaches, e.g., outperforming TransNetV2 by 4.2%, when being derived and evaluated on our newly constructed SHOT dataset. Moreover, to validate the generalizability of the AutoShot architecture, we directly evaluate it on another three public datasets: ClipShots, BBC and RAI, and the F1 scores of AutoShot outperform previous state-of-the-art approaches by 1.1%, 0.9% and 1.2%, respectively. The SHOT dataset and code can be found in https://github.com/wentaozhu/AutoShot.git .