# UBoCo : Unsupervised Boundary Contrastive Learning for Generic Event Boundary Detection

arXiv: https://arxiv.org/abs/2111.14799 · HTML: https://mlsanity.com/paper/2111.14799/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 45 references that are in the map.
- First posted on arXiv 29 Nov 2021.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [Spatiotemporal Contrastive Video Representation Learning](https://mlsanity.com/paper/2008.03800/) (2020)
- [BMN: Boundary-Matching Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1907.09702/) (2019)
- [BSN: Boundary Sensitive Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1806.02964/) (2018)
- [Parametric Contrastive Learning](https://mlsanity.com/paper/2107.12028/) (2021)
- [VideoMoCo: Contrastive Video Representation Learning with Temporally Adversarial Examples](https://mlsanity.com/paper/2103.05905/) (2021)
- [TURN TAP: Temporal Unit Regression Network for Temporal Action Proposals](https://mlsanity.com/paper/1703.06189/) (2017)
- [Segmental Spatiotemporal CNNs for Fine-grained Action Segmentation](https://mlsanity.com/paper/1602.02995/) (2016)
- [Counting Out Time: Class Agnostic Video Repetition Counting in the Wild](https://mlsanity.com/paper/2006.15418/) (2020)
- [Generic Event Boundary Detection: A Benchmark for Event Segmentation](https://mlsanity.com/paper/2101.10511/) (2021)
- [ViSiL: Fine-grained Spatio-Temporal Video Similarity Learning](https://mlsanity.com/paper/1908.07410/) (2019)
- [Intra- and Inter-Action Understanding via Temporal Action Parsing](https://mlsanity.com/paper/2005.10229/) (2020)
- [Shot Contrastive Self-Supervised Learning for Scene Boundary Detection](https://mlsanity.com/paper/2104.13537/) (2021)
- [Zero-shot Natural Language Video Localization](https://mlsanity.com/paper/2110.00428/) (2021)

## Built on by (most cited first, 20 of 20)

- [Knowing Where to Focus: Event-aware Transformer for Video Grounding](https://mlsanity.com/paper/2308.06947/) (2023)
- [Prior Knowledge Integration via LLM Encoding and Pseudo Event Regulation for Video Moment Retrieval](https://mlsanity.com/paper/2407.15051/) (2024)
- [Efficient and Effective Weakly-Supervised Action Segmentation via Action-Transition-Aware Boundary Alignment](https://mlsanity.com/paper/2403.19225/) (2024)
- [OTAS: Unsupervised Boundary Detection for Object-Centric Temporal Action Segmentation](https://mlsanity.com/paper/2309.06276/) (2023)
- [Structured Context Transformer for Generic Event Boundary Detection](https://mlsanity.com/paper/2206.02985/) (2022)
- [Rethinking the Architecture Design for Efficient Generic Event Boundary Detection](https://mlsanity.com/paper/2407.12622/) (2024)
- [Soft-Landing Strategy for Alleviating the Task Discrepancy Problem in Temporal Action Localization Tasks](https://mlsanity.com/paper/2211.06023/) (2022)
- [VVS: Video-to-Video Retrieval with Irrelevant Frame Suppression](https://mlsanity.com/paper/2303.08906/) (2023)
- [Online Generic Event Boundary Detection](https://mlsanity.com/paper/2510.06855/) (2025)
- [Event-aware Video Corpus Moment Retrieval](https://mlsanity.com/paper/2402.13566/) (2024)
- [What's in the Flow? Exploiting Temporal Motion Cues for Unsupervised Generic Event Boundary Detection](https://mlsanity.com/paper/2404.18935/) (2024)
- [Fine-grained Dynamic Network for Generic Event Boundary Detection](https://mlsanity.com/paper/2407.04274/) (2024)
- [Motion Aware Self-Supervision for Generic Event Boundary Detection](https://mlsanity.com/paper/2210.05574/) (2022)
- [Convex Combination Consistency between Neighbors for Weakly-supervised Action Localization](https://mlsanity.com/paper/2205.00400/) (2022)
- [Generic Event Boundary Detection in Video with Pyramid Features](https://mlsanity.com/paper/2301.04288/) (2023)
- [Generic Event Boundary Detection via Denoising Diffusion](https://mlsanity.com/paper/2508.12084/) (2025)
- [Structured Context Learning for Generic Event Boundary Detection](https://mlsanity.com/paper/2512.00475/) (2025)
- [IVAC-P2L: Leveraging Irregular Repetition Priors for Improving Video Action Counting](https://mlsanity.com/paper/2403.11959/) (2024)
- [Mitigating Modality and Language-Style Gaps for Zero-Shot Video Moment Retrieval](https://mlsanity.com/paper/2607.19027/) (2026)
- [VidParse: Online Parsing of Egocentric Procedures Like a Pro](https://mlsanity.com/paper/2608.27562/) (2026)

## Abstract (arXiv)

Generic Event Boundary Detection (GEBD) is a newly suggested video understanding task that aims to find one level deeper semantic boundaries of events. Bridging the gap between natural human perception and video understanding, it has various potential applications, including interpretable and semantically valid video parsing. Still at an early development stage, existing GEBD solvers are simple extensions of relevant video understanding tasks, disregarding GEBD's distinctive characteristics. In this paper, we propose a novel framework for unsupervised/supervised GEBD, by using the Temporal Self-similarity Matrix (TSM) as the video representation. The new Recursive TSM Parsing (RTP) algorithm exploits local diagonal patterns in TSM to detect boundaries, and it is combined with the Boundary Contrastive (BoCo) loss to train our encoder to generate more informative TSMs. Our framework can be applied to both unsupervised and supervised settings, with both achieving state-of-the-art performance by a huge margin in GEBD benchmark. Especially, our unsupervised method outperforms the previous state-of-the-art "supervised" model, implying its exceptional efficacy.