# Unsupervised View-Invariant Human Posture Representation

arXiv: https://arxiv.org/abs/2109.08730 · HTML: https://mlsanity.com/paper/2109.08730/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 47 references that are in the map.
- First posted on arXiv 17 Sep 2021.

## Builds on (most cited first, 30 of 30)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognition](https://mlsanity.com/paper/1904.12659/) (2019)
- [Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images](https://mlsanity.com/paper/1804.04488/) (2018)
- [Towards Long-Form Video Understanding](https://mlsanity.com/paper/2106.11310/) (2021)
- [Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition](https://mlsanity.com/paper/1904.01189/) (2019)
- [A Variational U-Net for Conditional Appearance and Shape Generation](https://mlsanity.com/paper/1804.04694/) (2018)
- [What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment](https://mlsanity.com/paper/1904.04346/) (2019)
- [Unsupervised 3D Pose Estimation with Geometric Self-Supervision](https://mlsanity.com/paper/1904.04812/) (2019)
- [MS2L: Multi-Task Self-Supervised Learning for Skeleton Based Action Recognition](https://mlsanity.com/paper/2010.05599/) (2020)
- [View Adaptive Neural Networks for High Performance Skeleton-based Human Action Recognition](https://mlsanity.com/paper/1804.07453/) (2018)
- [Uncertainty-aware Score Distribution Learning for Action Quality Assessment](https://mlsanity.com/paper/2006.07665/) (2020)
- [Unsupervised Geometry-Aware Representation for 3D Human Pose Estimation](https://mlsanity.com/paper/1804.01110/) (2018)
- [Learning Monocular 3D Human Pose Estimation from Multi-view Images](https://mlsanity.com/paper/1803.04775/) (2018)
- [The Pros and Cons: Rank-aware Temporal Attention for Skill Determination in Long Videos](https://mlsanity.com/paper/1812.05538/) (2018)
- [Augmented Skeleton Based Contrastive Action Learning with Momentum LSTM for Unsupervised Action Recognition](https://mlsanity.com/paper/2008.00188/) (2020)
- [VPN: Learning Video-Pose Embedding for Activities of Daily Living](https://mlsanity.com/paper/2007.03056/) (2020)
- [Synthetic Humans for Action Recognition from Unseen Viewpoints](https://mlsanity.com/paper/1912.04070/) (2019)
- [Unsupervised Learning of Long-Term Motion Dynamics for Videos](https://mlsanity.com/paper/1701.01821/) (2017)
- [Chained Multi-stream Networks Exploiting Pose, Motion, and Appearance for Action Classification and Detection](https://mlsanity.com/paper/1704.00616/) (2017)
- [Unsupervised Learning of View-invariant Action Representations](https://mlsanity.com/paper/1809.01844/) (2018)
- [Weakly-Supervised Discovery of Geometry-Aware Representation for 3D Human Pose Estimation](https://mlsanity.com/paper/1903.08839/) (2019)
- [View-invariant Deep Architecture for Human Action Recognition using late fusion](https://mlsanity.com/paper/1912.03632/) (2019)
- [Unsupervised Disentanglement of Pose, Appearance and Background from Images and Videos](https://mlsanity.com/paper/2001.09518/) (2020)
- [Learning a Deep Model for Human Action Recognition from Novel Viewpoints](https://mlsanity.com/paper/1602.00828/) (2016)
- [VI-Net: View-Invariant Quality of Human Movement Assessment](https://mlsanity.com/paper/2008.04999/) (2020)

## Built on by (most cited first, 3 of 3)

- [ViA: View-invariant Skeleton Action Representation Learning via Motion Retargeting](https://mlsanity.com/paper/2209.00065/) (2022)
- [SHARE: Single-view Human Adversarial REconstruction](https://mlsanity.com/paper/2401.00343/) (2023)
- [ViewBridge: Curriculum Knowledge Distillation for Activity View-Invariance Under Extreme Viewpoint Changes](https://mlsanity.com/paper/2504.05451/) (2025)

## Abstract (arXiv)

Most recent view-invariant action recognition and performance assessment approaches rely on a large amount of annotated 3D skeleton data to extract view-invariant features. However, acquiring 3D skeleton data can be cumbersome, if not impractical, in in-the-wild scenarios. To overcome this problem, we present a novel unsupervised approach that learns to extract view-invariant 3D human pose representation from a 2D image without using 3D joint data. Our model is trained by exploiting the intrinsic view-invariant properties of human pose between simultaneous frames from different viewpoints and their equivariant properties between augmented frames from the same viewpoint. We evaluate the learned view-invariant pose representations for two downstream tasks. We perform comparative experiments that show improvements on the state-of-the-art unsupervised cross-view action classification accuracy on NTU RGB+D by a significant margin, on both RGB and depth images. We also show the efficiency of transferring the learned representations from NTU RGB+D to obtain the first ever unsupervised cross-view and cross-subject rank correlation results on the multi-view human movement quality dataset, QMAR, and marginally improve on the-state-of-the-art supervised results for this dataset. We also carry out ablation studies to examine the contributions of the different components of our proposed network.