# Cross-Model Pseudo-Labeling for Semi-Supervised Action Recognition

arXiv: https://arxiv.org/abs/2112.09690 · HTML: https://mlsanity.com/paper/2112.09690/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 47 references that are in the map.
- First posted on arXiv 17 Dec 2021.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification](https://mlsanity.com/paper/1712.04851/) (2017)
- [Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks](https://mlsanity.com/paper/1711.10305/) (2017)
- [Convolutional Two-Stream Network Fusion for Video Action Recognition](https://mlsanity.com/paper/1604.06573/) (2016)
- [S4L: Self-Supervised Semi-Supervised Learning](https://mlsanity.com/paper/1905.03670/) (2019)
- [Temporal Relational Reasoning in Videos](https://mlsanity.com/paper/1711.08496/) (2017)
- [Self-supervised Co-training for Video Representation Learning](https://mlsanity.com/paper/2010.09709/) (2020)
- [A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning](https://mlsanity.com/paper/2104.14558/) (2021)
- [Video Representation Learning by Dense Predictive Coding](https://mlsanity.com/paper/1909.04656/) (2019)
- [SpeedNet: Learning the Speediness in Videos](https://mlsanity.com/paper/2004.06130/) (2020)
- [Temporal Pyramid Network for Action Recognition](https://mlsanity.com/paper/2004.03548/) (2020)
- [Self-Supervised Video Representation Learning with Space-Time Cubic Puzzles](https://mlsanity.com/paper/1811.09795/) (2018)
- [Self-supervised Video Representation Learning by Pace Prediction](https://mlsanity.com/paper/2008.05861/) (2020)
- [Self-supervised Spatio-temporal Representation Learning for Videos by Predicting Motion and Appearance Statistics](https://mlsanity.com/paper/1904.03597/) (2019)
- [Action Recognition with Trajectory-Pooled Deep-Convolutional Descriptors](https://mlsanity.com/paper/1505.04868/) (2015)
- [Video Representation Learning with Visual Tempo Consistency](https://mlsanity.com/paper/2006.15489/) (2020)
- [Semi-Supervised Action Recognition with Temporal Contrastive Learning](https://mlsanity.com/paper/2102.02751/) (2021)
- [Multiview Pseudo-Labeling for Semi-supervised Learning from Video](https://mlsanity.com/paper/2104.00682/) (2021)

## Built on by (most cited first, 29 of 29)

- [Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2208.09910/) (2022)
- [Unsupervised Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2204.03649/) (2022)
- [SVFormer: Semi-supervised Video Transformer for Action Recognition](https://mlsanity.com/paper/2211.13222/) (2022)
- [UniMatch V2: Pushing the Limit of Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2410.10777/) (2024)
- [Frame-wise Action Representations for Long Videos via Sequence Contrastive Learning](https://mlsanity.com/paper/2203.14957/) (2022)
- [Semi-Supervised Deep Regression with Uncertainty Consistency and Variational Model Ensembling via Bayesian Neural Networks](https://mlsanity.com/paper/2302.07579/) (2023)
- [TimeBalance: Temporally-Invariant and Temporally-Distinctive Video Representations for Semi-Supervised Action Recognition](https://mlsanity.com/paper/2303.16268/) (2023)
- [Cyclical Self-Supervision for Semi-Supervised Ejection Fraction Prediction from Echocardiogram Videos](https://mlsanity.com/paper/2210.11291/) (2022)
- [Semi-Supervised Semantic Segmentation Based on Pseudo-Labels: A Survey](https://mlsanity.com/paper/2403.01909/) (2024)
- [STiL: Semi-supervised Tabular-Image Learning for Comprehensive Task-Relevant Information Exploration in Multimodal Classification](https://mlsanity.com/paper/2503.06277/) (2025)
- [Two-shot Video Object Segmentation](https://mlsanity.com/paper/2303.12078/) (2023)
- [Knowledge Guided Semi-Supervised Learning for Quality Assessment of User Generated Videos](https://mlsanity.com/paper/2312.15425/) (2023)
- [Learning Articulated Shape with Keypoint Pseudo-labels from Web Images](https://mlsanity.com/paper/2304.14396/) (2023)
- [EC-Depth: Exploring the consistency of self-supervised monocular depth estimation in challenging scenes](https://mlsanity.com/paper/2310.08044/) (2023)
- [Language Semantic Graph Guided Data-Efficient Learning](https://mlsanity.com/paper/2311.08782/) (2023)
- [ActNetFormer: Transformer-ResNet Hybrid Method for Semi-Supervised Action Recognition in Videos](https://mlsanity.com/paper/2404.06243/) (2024)
- [FinePseudo: Improving Pseudo-Labelling through Temporal-Alignablity for Semi-Supervised Fine-Grained Action Recognition](https://mlsanity.com/paper/2409.01448/) (2024)
- [Dual Invariance Self-training for Reliable Semi-supervised Surgical Phase Recognition](https://mlsanity.com/paper/2501.17628/) (2025)
- [On the Surprising Effectiveness of Transformers in Low-Labeled Video Recognition](https://mlsanity.com/paper/2209.07474/) (2022)
- [Learning Discriminative Spatio-temporal Representations for Semi-supervised Action Recognition](https://mlsanity.com/paper/2404.16416/) (2024)
- [SemiVT-Surge: Semi-Supervised Video Transformer for Surgical Phase Recognition](https://mlsanity.com/paper/2506.01471/) (2025)
- [Active Learning on a Programmable Photonic Quantum Processor](https://mlsanity.com/paper/2208.02104/) (2022)
- [Refining Action Boundaries for One-stage Detection](https://mlsanity.com/paper/2210.14284/) (2022)
- [Self-supervised and Weakly Supervised Contrastive Learning for Frame-wise Action Representations](https://mlsanity.com/paper/2212.03125/) (2022)
- [Egocentric RGB+Depth Action Recognition in Industry-Like Settings](https://mlsanity.com/paper/2309.13962/) (2023)
- [SITAR: Semi-supervised Image Transformer for Action Recognition](https://mlsanity.com/paper/2409.02910/) (2024)
- [Semi-Supervised Audio-Visual Video Action Recognition with Audio Source Localization Guided Mixup](https://mlsanity.com/paper/2503.02284/) (2025)
- [Data-Efficient Learning for Generalizable Surgical Video Understanding](https://mlsanity.com/paper/2508.10215/) (2025)
- [LEViL: Label-Efficient Video Learning via Zero-Shot Distillation over VLM-Generated Pseudo-Label Spaces](https://mlsanity.com/paper/2606.21358/) (2026)

## Abstract (arXiv)

Semi-supervised action recognition is a challenging but important task due to the high cost of data annotation. A common approach to this problem is to assign unlabeled data with pseudo-labels, which are then used as additional supervision in training. Typically in recent work, the pseudo-labels are obtained by training a model on the labeled data, and then using confident predictions from the model to teach itself. In this work, we propose a more effective pseudo-labeling scheme, called Cross-Model Pseudo-Labeling (CMPL). Concretely, we introduce a lightweight auxiliary network in addition to the primary backbone, and ask them to predict pseudo-labels for each other. We observe that, due to their different structural biases, these two models tend to learn complementary representations from the same video clips. Each model can thus benefit from its counterpart by utilizing cross-model predictions as supervision. Experiments on different data partition protocols demonstrate the significant improvement of our framework over existing alternatives. For example, CMPL achieves 17.6% and 25.1% Top-1 accuracy on Kinetics-400 and UCF-101 using only the RGB modality and 1% labeled data, outperforming our baseline model, FixMatch, by 9.0% and 10.3%, respectively.