# COMEDIAN: Self-Supervised Learning and Knowledge Distillation for Action Spotting using Transformers

arXiv: https://arxiv.org/abs/2309.01270 · HTML: https://mlsanity.com/paper/2309.01270/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 56 of its 66 references that are in the map.
- First posted on arXiv 3 Sep 2023.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)

## Builds on (most cited first, 30 of 56)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019)
- [VideoBERT: A Joint Model for Video and Language Representation Learning](https://mlsanity.com/paper/1904.01766/) (2019)
- [Masked Feature Prediction for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2112.09133/) (2021)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [Masked Autoencoders As Spatiotemporal Learners](https://mlsanity.com/paper/2205.09113/) (2022)
- [Spatiotemporal Contrastive Video Representation Learning](https://mlsanity.com/paper/2008.03800/) (2020)
- [With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2104.14548/) (2021)
- [Video Transformer Network](https://mlsanity.com/paper/2102.00719/) (2021)
- [A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning](https://mlsanity.com/paper/2104.14558/) (2021)

## Built on by (most cited first, 17 of 17)

- [X-VARS: Introducing Explainability in Football Refereeing with Multi-Modal Large Language Model](https://mlsanity.com/paper/2404.06332/) (2024)
- [T-DEED: Temporal-Discriminability Enhancer Encoder-Decoder for Precise Event Spotting in Sports Videos](https://mlsanity.com/paper/2404.05392/) (2024)
- [Action Anticipation from SoccerNet Football Video Broadcasts](https://mlsanity.com/paper/2504.12021/) (2025)
- [Survey of Action Recognition, Spotting and Spatio-Temporal Localization in Soccer -- Current Trends and Research Perspectives](https://mlsanity.com/paper/2309.12067/) (2023)
- [Deep learning for action spotting in association football videos](https://mlsanity.com/paper/2410.01304/) (2024)
- [Precise Event Spotting in Sports Videos: Solving Long-Range Dependency and Class Imbalance](https://mlsanity.com/paper/2503.00147/) (2025)
- [Deep Learning for Sports Video Event Detection: Tasks, Datasets, Methods, and Challenges](https://mlsanity.com/paper/2505.03991/) (2025)
- [OSL-ActionSpotting: A Unified Library for Action Spotting in Sports Videos](https://mlsanity.com/paper/2407.01265/) (2024)
- [AdaSpot: Spend Resolution Where It Matters for Precise Event Spotting](https://mlsanity.com/paper/2602.22073/) (2026)
- [Detecting Precise Hand Touch Moments in Egocentric Video](https://mlsanity.com/paper/2604.12343/) (2026)
- [Investigating Event-Based Cameras for Video Frame Interpolation in Sports](https://mlsanity.com/paper/2407.02370/) (2024)
- [Learning from Random Subspace Exploration: Generalized Test-Time Augmentation with Self-supervised Distillation](https://mlsanity.com/paper/2507.01347/) (2025)
- [Multi-Focus Temporal Shifting for Precise Event Spotting in Sports Videos](https://mlsanity.com/paper/2507.07381/) (2025)
- [Decompose, Mix, Adapt: A Unified Framework for Parameter-Efficient Neural Network Recombination and Compression](https://mlsanity.com/paper/2603.27383/) (2026)
- [Denoise and Align: Diffusion-Driven Foreground Knowledge Prompting for Open-Vocabulary Temporal Action Detection](https://mlsanity.com/paper/2604.18313/) (2026)
- [Entity-Aware Sequence Transduction for Player-Centric Ball Action Spotting](https://mlsanity.com/paper/2608.01696/) (2026)
- [A Survey of Large Models in Sports](https://mlsanity.com/paper/2608.14377/) (2026)

## Abstract (arXiv)

We present COMEDIAN, a novel pipeline to initialize spatiotemporal transformers for action spotting, which involves self-supervised learning and knowledge distillation. Action spotting is a timestamp-level temporal action detection task. Our pipeline consists of three steps, with two initialization stages. First, we perform self-supervised initialization of a spatial transformer using short videos as input. Additionally, we initialize a temporal transformer that enhances the spatial transformer's outputs with global context through knowledge distillation from a pre-computed feature bank aligned with each short video segment. In the final step, we fine-tune the transformers to the action spotting task. The experiments, conducted on the SoccerNet-v2 dataset, demonstrate state-of-the-art performance and validate the effectiveness of COMEDIAN's pretraining paradigm. Our results highlight several advantages of our pretraining pipeline, including improved performance and faster convergence compared to non-pretrained models.