mlsanityData to 7 Oct 2026

Scene Consistency Representation Learning for Video Scene Segmentation

arXiv 2205.05487 ↗First posted 11 May 2022cs.CV cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

A long-term video, such as a movie or TV show, is composed of various scenes, each of which represents a series of shots sharing the same semantic story. Spotting the correct scene boundary from the long-term video is a challenging task, since a model must understand the storyline of the video to figure out where a scene starts and ends. To this end, we propose an effective Self-Supervised Learning (SSL) framework to learn better shot representations from unlabeled long-term videos. More specifically, we present an SSL scheme to achieve scene consistency, while exploring considerable data augmentation and shuffling methods to boost the model generalizability. Instead of explicitly learning the scene boundary features as in the previous methods, we introduce a vanilla temporal model with less inductive bias to verify the quality of the shot features. Our method achieves the state-of-the-art performance on the task of Video Scene Segmentation. Additionally, we suggest a more fair and reasonable benchmark to evaluate the performance of Video Scene Segmentation methods. The code is made available.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 33 of its 45 references that are in the map. First posted on arXiv 11 May 2022.

Builds on

33 of 45 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  6. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  7. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  8. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  9. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  10. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  11. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  12. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
21 more
  1. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  2. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  3. 2016Colorful Image Colorizationcited by 1,113
  4. 2015Bidirectional LSTM-CRF Models for Sequence Taggingcited by 544
  5. 2019End-to-End Learning of Visual Representations from Uncurated Instructional Videoscited by 506
  6. 2020Spatiotemporal Contrastive Video Representation Learningcited by 372
  7. 2020Self-Supervised MultiModal Versatile Networkscited by 276
  8. 2018BSN: Boundary Sensitive Network for Temporal Action Proposal Generationcited by 271
  9. 2021A Large-Scale Study on Unsupervised Spatiotemporal Representation Learningcited by 214
  10. 2020MovieNet: A Holistic Dataset for Movie Understandingcited by 206
  11. 2018SoccerNet: A Scalable Dataset for Action Spotting in Soccer Videoscited by 148
  12. 2021Relaxed Transformer Decoders for Direct Action Proposal Generationcited by 116
  13. 2021Broaden Your Views for Self-Supervised Video Learningcited by 108
  14. 2021Rethinking Self-supervised Correspondence Learning: A Video Frame-level Similarity Perspectivecited by 81
  15. 2021Unsupervised Object-Level Representation Learning from Scene Imagescited by 72
  16. 2020A Local-to-Global Approach to Multi-modal Movie Scene Segmentationcited by 60
  17. 2019A Context-Aware Loss Function for Action Spotting in Soccer Videoscited by 56
  18. 2021Video Contrastive Learning with Global Contextcited by 40
  19. 2021Shot Contrastive Self-Supervised Learning for Scene Boundary Detectioncited by 35
  20. 2015A Deep Siamese Network for Scene Detection in Broadcast Videoscited by 21
  21. 2021Group-wise Inhibition based Feature Regularization for Robust Classificationcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 3; 2026: 2.

9 papers.

  1. 2025MultiShotMaster: A Controllable Multi-Shot Video Generation Frameworkcited by 33
  2. 2025Video Color Grading via Look-Up Table Generationcited by 9
  3. 2023Collaborative Noisy Label Cleaner: Learning Scene-aware Trailers for Multi-modal Highlight Detection in Moviescited by 4
  4. 2023MEGA: Multimodal Alignment Aggregation and Distillation For Cinematic Video Segmentationcited by 4
  5. 2025Scene-VLM: Multimodal Video Scene Segmentation via Vision-Language Modelscited by 2
  6. 2024Generating Event-oriented Attribution for Movies via Two-Stage Prefix-Enhanced Multimodal LLMcited by 0
  7. 2024Modality-Aware Shot Relating and Comparing for Video Scene Detectioncited by 0
  8. 2026MAVIN: Multi-Shot Audio-Visual Generation with Customized Narrative Controlcited by 0
  9. 2026Beyond Visual Boundaries: Rethinking Scene Segmentation for Movie RAGcited by 0