# Learning View-Disentangled Human Pose Representation by Contrastive Cross-View Mutual Information Maximization

arXiv: https://arxiv.org/abs/2012.01405 · HTML: https://mlsanity.com/paper/2012.01405/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Dec 2020.

## Built on by (most cited first, 19 of 19)

- [Multimodal Variational Auto-encoder based Audio-Visual Segmentation](https://mlsanity.com/paper/2310.08303/) (2023)
- [Large Language Models are Efficient Learners of Noise-Robust Speech Recognition](https://mlsanity.com/paper/2401.10446/) (2024)
- [Video Pose Distillation for Few-Shot, Fine-Grained Sports Action Recognition](https://mlsanity.com/paper/2109.01305/) (2021)
- [Temporal Feature Alignment and Mutual Information Maximization for Video-Based Human Pose Estimation](https://mlsanity.com/paper/2203.15227/) (2022)
- [ViA: View-invariant Skeleton Action Representation Learning via Motion Retargeting](https://mlsanity.com/paper/2209.00065/) (2022)
- [Learning from Semantic Alignment between Unpaired Multiviews for Egocentric Video Recognition](https://mlsanity.com/paper/2308.11489/) (2023)
- [COMPOSER: Compositional Reasoning of Group Activity in Videos with Keypoint-Only Modality](https://mlsanity.com/paper/2112.05892/) (2021)
- [DVANet: Disentangling View and Action Features for Multi-View Action Recognition](https://mlsanity.com/paper/2312.05719/) (2023)
- [Mutual Information-Based Temporal Difference Learning for Human Pose Estimation in Video](https://mlsanity.com/paper/2303.08475/) (2023)
- [MoCaNet: Motion Retargeting in-the-wild via Canonicalization Networks](https://mlsanity.com/paper/2112.10082/) (2021)
- [Dessie: Disentanglement for Articulated 3D Horse Shape and Pose Estimation from Images](https://mlsanity.com/paper/2410.03438/) (2024)
- [Learning Disentangled Representation with Mutual Information Maximization for Real-Time UAV Tracking](https://mlsanity.com/paper/2308.10262/) (2023)
- [Collaborating Domain-shared and Target-specific Feature Clustering for Cross-domain 3D Action Recognition](https://mlsanity.com/paper/2207.09767/) (2022)
- [Deep video representation learning: a survey](https://mlsanity.com/paper/2405.06574/) (2024)
- [Disentangled Representation Learning with Transmitted Information Bottleneck](https://mlsanity.com/paper/2311.01686/) (2023)
- [Graph Contrastive Learning with Cross-view Reconstruction](https://mlsanity.com/paper/2209.07699/) (2022)
- [Multi-View Clustering from the Perspective of Mutual Information](https://mlsanity.com/paper/2302.08743/) (2023)
- [VIFSS: View-Invariant and Figure Skating-Specific Pose Representation Learning for Temporal Action Segmentation](https://mlsanity.com/paper/2508.10281/) (2025)
- [Cross-Domain Human Action Recognition from Multiview Motion and Textual Descriptions](https://mlsanity.com/paper/2605.22697/) (2026)

## Abstract (arXiv)

We introduce a novel representation learning method to disentangle pose-dependent as well as view-dependent factors from 2D human poses. The method trains a network using cross-view mutual information maximization (CV-MIM) which maximizes mutual information of the same pose performed from different viewpoints in a contrastive learning manner. We further propose two regularization terms to ensure disentanglement and smoothness of the learned representations. The resulting pose representations can be used for cross-view action recognition. To evaluate the power of the learned representations, in addition to the conventional fully-supervised action recognition settings, we introduce a novel task called single-shot cross-view action recognition. This task trains models with actions from only one single viewpoint while models are evaluated on poses captured from all possible viewpoints. We evaluate the learned representations on standard benchmarks for action recognition, and show that (i) CV-MIM performs competitively compared with the state-of-the-art models in the fully-supervised scenarios; (ii) CV-MIM outperforms other competing methods by a large margin in the single-shot cross-view setting; (iii) and the learned representations can significantly boost the performance when reducing the amount of supervised training data. Our code is made publicly available at https://github.com/google-research/google-research/tree/master/poem