Motion Aware Self-Supervision for Generic Event Boundary Detection
Not on the map. Why?
Abstract (arXiv)
The task of Generic Event Boundary Detection (GEBD) aims to detect moments in videos that are naturally perceived by humans as generic and taxonomy-free event boundaries. Modeling the dynamically evolving temporal and spatial changes in a video makes GEBD a difficult problem to solve. Existing approaches involve very complex and sophisticated pipelines in terms of architectural design choices, hence creating a need for more straightforward and simplified approaches. In this work, we address this issue by revisiting a simple and effective self-supervised method and augment it with a differentiable motion feature learning module to tackle the spatial and temporal diversities in the GEBD task. We perform extensive experiments on the challenging Kinetics-GEBD and TAPOS datasets to demonstrate the efficacy of the proposed approach compared to the other self-supervised state-of-the-art methods. We also show that this simple self-supervised approach learns motion features without any explicit motion-specific pretext task.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 67 of its 84 references that are in the map. First posted on arXiv 11 Oct 2022.
Builds on
67 of 84 references.
References · 67
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
- 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
- 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
- 2017The Kinetics Human Action Video Datasetcited by 1,894
- 2018SlowFast Networks for Video Recognitioncited by 1,862
- 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
- 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
- 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
- 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
38 more
- 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
- 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
- 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
- 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
- 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 852
- 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
- 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 780
- 2019Self-labelling via simultaneous clustering and representation learningcited by 525
- 2017Future Frame Prediction for Anomaly Detection -- A New Baselinecited by 391
- 2016Convolutional Two-Stream Network Fusion for Video Action Recognitioncited by 384
- 2019BMN: Boundary-Matching Network for Temporal Action Proposal Generationcited by 327
- 2018BSN: Boundary Sensitive Network for Temporal Action Proposal Generationcited by 271
- 2017Unsupervised Representation Learning by Sorting Sequencescited by 219
- 2018Rethinking the Faster R-CNN Architecture for Temporal Action Localizationcited by 194
- 2019Video Representation Learning by Dense Predictive Codingcited by 175
- 2019STM: SpatioTemporal and Motion Encoding for Action Recognitioncited by 142
- 2020Self-supervised Video Representation Learning by Pace Predictioncited by 139
- 2017Convolutional neural network architecture for geometric matchingcited by 138
- 2020Memory-augmented Dense Predictive Coding for Video Representation Learningcited by 137
- 2021TCLR: Temporal Contrastive Learning for Video Representationcited by 136
- 2018When will you do what? - Anticipating Temporal Occurrences of Activitiescited by 106
- 2022Self-Supervised Learning for Videos: A Surveycited by 85
- 2017Optical Flow Guided Feature: A Fast and Robust Motion Representation for Video Action Recognitioncited by 82
- 2016Segmental Spatiotemporal CNNs for Fine-grained Action Segmentationcited by 78
- 2018Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignmentcited by 74
- 2020MotionSqueeze: Neural Motion Feature Learning for Video Understandingcited by 72
- 2018Self-Supervised Spatiotemporal Feature Learning via Video Rotation Predictioncited by 71
- 2016Connectionist Temporal Modeling for Weakly Supervised Action Labelingcited by 70
- 2021Generic Event Boundary Detection: A Benchmark for Event Segmentationcited by 64
- 2019SFNet: Learning Object-aware Semantic Correspondencecited by 60
- 2020Video Representation Learning with Visual Tempo Consistencycited by 57
- 2019Hyperpixel Flow: Semantic Correspondence with Multi-layer Neural Featurescited by 56
- 2018Representation Flow for Action Recognitioncited by 51
- 2017SCNet: Learning Semantic Correspondencecited by 48
- 2020Self-supervised Video Representation Learning Using Inter-intra Contrastive Frameworkcited by 46
- 2018End-to-End Learning of Motion Representation for Video Understandingcited by 43
- 2020Intra- and Inter-Action Understanding via Temporal Action Parsingcited by 41
- 2021Video Contrastive Learning with Global Contextcited by 40
50 most cited of 67 All 67 (JSON)
Built on by
Citing papers per year: 2024: 1; 2025: 2.
3 papers.