# ViewCLR: Learning Self-supervised Video Representation for Unseen Viewpoints

arXiv: https://arxiv.org/abs/2112.03905 · HTML: https://mlsanity.com/paper/2112.03905/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 71 references that are in the map.
- First posted on arXiv 7 Dec 2021.

## Builds on (most cited first, 30 of 45)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?](https://mlsanity.com/paper/1711.09577/) (2017)
- [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019)
- [Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification](https://mlsanity.com/paper/1712.04851/) (2017)
- [End-to-End Learning of Visual Representations from Uncurated Instructional Videos](https://mlsanity.com/paper/1912.06430/) (2019)
- [Look, Listen and Learn](https://mlsanity.com/paper/1705.08168/) (2017)
- [Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2003.14111/) (2020)
- [Audio-Visual Scene Analysis with Self-Supervised Multisensory Features](https://mlsanity.com/paper/1804.03641/) (2018)
- [With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2104.14548/) (2021)
- [Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2107.12213/) (2021)
- [Learning Correspondence from the Cycle-Consistency of Time](https://mlsanity.com/paper/1903.07593/) (2019)
- [Objects that Sound](https://mlsanity.com/paper/1712.06651/) (2017)
- [Self-supervised Co-training for Video Representation Learning](https://mlsanity.com/paper/2010.09709/) (2020)
- [Unsupervised Representation Learning by Sorting Sequences](https://mlsanity.com/paper/1708.01246/) (2017)
- [A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning](https://mlsanity.com/paper/2104.14558/) (2021)
- [Tracking Emerges by Colorizing Videos](https://mlsanity.com/paper/1806.09594/) (2018)
- [Cross-view Action Modeling, Learning and Recognition](https://mlsanity.com/paper/1405.2941/) (2014)
- [Broaden Your Views for Self-Supervised Video Learning](https://mlsanity.com/paper/2103.16559/) (2021)
- [3D Human Action Representation Learning via Cross-View Consistency Pursuit](https://mlsanity.com/paper/2104.14466/) (2021)
- [View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Data](https://mlsanity.com/paper/1703.08274/) (2017)
- [Skeleton-Based Action Recognition with Multi-Stream Adaptive Graph Convolutional Networks](https://mlsanity.com/paper/1912.06971/) (2019)

## Built on by (most cited first, 20 of 20)

- [Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?](https://mlsanity.com/paper/2206.05266/) (2022)
- [Just Add π! Pose Induced Video Transformers for Understanding Activities of Daily Living](https://mlsanity.com/paper/2311.18840/) (2023)
- [DVANet: Disentangling View and Action Features for Multi-View Action Recognition](https://mlsanity.com/paper/2312.05719/) (2023)
- [Hypergraph-based Multi-View Action Recognition using Event Cameras](https://mlsanity.com/paper/2403.19316/) (2024)
- [Learning Viewpoint-Agnostic Visual Representations by Recovering Tokens in 3D Space](https://mlsanity.com/paper/2206.11895/) (2022)
- [STEPs: Self-Supervised Key Step Extraction and Localization from Unlabeled Procedural Videos](https://mlsanity.com/paper/2301.00794/) (2023)
- [POV: Prompt-Oriented View-Agnostic Learning for Egocentric Hand-Object Interaction in the Multi-View World](https://mlsanity.com/paper/2403.05856/) (2024)
- [MV2MAE: Multi-View Video Masked Autoencoders](https://mlsanity.com/paper/2401.15900/) (2024)
- [KAN-HyperpointNet for Point Cloud Sequence-Based 3D Human Action Recognition](https://mlsanity.com/paper/2409.09444/) (2024)
- [Probing Fine-Grained Action Understanding and Cross-View Generalization of Foundation Models](https://mlsanity.com/paper/2407.15605/) (2024)
- [Content-Distortion High-Order Interaction for Blind Image Quality Assessment](https://mlsanity.com/paper/2504.05076/) (2025)
- [Dual-view Spatio-Temporal Feature Fusion with CNN-Transformer Hybrid Network for Chinese Isolated Sign Language Recognition](https://mlsanity.com/paper/2506.06966/) (2025)
- [Trunk-branch Contrastive Network with Multi-view Deformable Aggregation for Multi-view Action Recognition](https://mlsanity.com/paper/2502.16493/) (2025)
- [VET-DINO: Learning Anatomical Understanding Through Multi-View Distillation in Veterinary Imaging](https://mlsanity.com/paper/2505.15248/) (2025)
- [View-aware Cross-modal Distillation for Multi-view Action Recognition](https://mlsanity.com/paper/2511.12870/) (2025)
- [Towards Generalizing Temporal Action Segmentation to Unseen Views](https://mlsanity.com/paper/2504.02512/) (2025)
- [ViewBridge: Curriculum Knowledge Distillation for Activity View-Invariance Under Extreme Viewpoint Changes](https://mlsanity.com/paper/2504.05451/) (2025)
- [Probabilistic Temporal Masked Attention for Cross-view Online Action Detection](https://mlsanity.com/paper/2508.17025/) (2025)
- [Resource-Efficient RGB-Only Action Recognition for Edge Deployment](https://mlsanity.com/paper/2602.10818/) (2026)
- [SHANDS: A Multi-View Dataset and Benchmark for Surgical Hand-Gesture and Error Recognition Toward Medical Training](https://mlsanity.com/paper/2603.26400/) (2026)

## Abstract (arXiv)

Learning self-supervised video representation predominantly focuses on discriminating instances generated from simple data augmentation schemes. However, the learned representation often fails to generalize over unseen camera viewpoints. To this end, we propose ViewCLR, that learns self-supervised video representation invariant to camera viewpoint changes. We introduce a view-generator that can be considered as a learnable augmentation for any self-supervised pre-text tasks, to generate latent viewpoint representation of a video. ViewCLR maximizes the similarities between the latent viewpoint representation with its representation from the original viewpoint, enabling the learned video encoder to generalize over unseen camera viewpoints. Experiments on cross-view benchmark datasets including NTU RGB+D dataset show that ViewCLR stands as a state-of-the-art viewpoint invariant self-supervised method.