# View-Invariant Probabilistic Embedding for Human Pose

arXiv: https://arxiv.org/abs/1912.01001 · HTML: https://mlsanity.com/paper/1912.01001/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Dec 2019.

## Built on by (most cited first, 30 of 47)

- [Probabilistic Embeddings for Cross-Modal Retrieval](https://mlsanity.com/paper/2101.05068/) (2021)
- [SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decomposition](https://mlsanity.com/paper/2106.03849/) (2021)
- [UATVR: Uncertainty-Adaptive Text-Video Retrieval](https://mlsanity.com/paper/2301.06309/) (2023)
- [Learning Fine-grained View-Invariant Representations from Unpaired Ego-Exo Videos via Temporal Alignment](https://mlsanity.com/paper/2306.05526/) (2023)
- [STAR-Transformer: A Spatio-temporal Cross Attention Transformer for Human Action Recognition](https://mlsanity.com/paper/2210.07503/) (2022)
- [MAP: Multimodal Uncertainty-Aware Vision-Language Pre-training Model](https://mlsanity.com/paper/2210.05335/) (2022)
- [Improved Probabilistic Image-Text Representations](https://mlsanity.com/paper/2305.18171/) (2023)
- [UNIK: A Unified Framework for Real-world Skeleton-based Action Recognition](https://mlsanity.com/paper/2107.08580/) (2021)
- [ElePose: Unsupervised 3D Human Pose Estimation by Predicting Camera Elevation and Learning Normalizing Flows on 2D Poses](https://mlsanity.com/paper/2112.07088/) (2021)
- [Probabilistic Representations for Video Contrastive Learning](https://mlsanity.com/paper/2204.03946/) (2022)
- [ExACT: Language-guided Conceptual Reasoning and Uncertainty Estimation for Event-based Action Recognition and More](https://mlsanity.com/paper/2403.12534/) (2024)
- [The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions](https://mlsanity.com/paper/2104.02710/) (2021)
- [Video Pose Distillation for Few-Shot, Fine-Grained Sports Action Recognition](https://mlsanity.com/paper/2109.01305/) (2021)
- [Generalizable Human Pose Triangulation](https://mlsanity.com/paper/2110.00280/) (2021)
- [Self-Supervised Keypoint Discovery in Behavioral Videos](https://mlsanity.com/paper/2112.05121/) (2021)
- [ViA: View-invariant Skeleton Action Representation Learning via Motion Retargeting](https://mlsanity.com/paper/2209.00065/) (2022)
- [Cross-Modal Learning with 3D Deformable Attention for Action Recognition](https://mlsanity.com/paper/2212.05638/) (2022)
- [COMPOSER: Compositional Reasoning of Group Activity in Videos with Keypoint-Only Modality](https://mlsanity.com/paper/2112.05892/) (2021)
- [Robust Multimodal Learning via Representation Decoupling](https://mlsanity.com/paper/2407.04458/) (2024)
- [Probabilistic Language-Image Pre-Training](https://mlsanity.com/paper/2410.18857/) (2024)
- [Learning Viewpoint-Agnostic Visual Representations by Recovering Tokens in 3D Space](https://mlsanity.com/paper/2206.11895/) (2022)
- [RICA2: Rubric-Informed, Calibrated Assessment of Actions](https://mlsanity.com/paper/2408.02138/) (2024)
- [FenceNet: Fine-grained Footwork Recognition in Fencing](https://mlsanity.com/paper/2204.09434/) (2022)
- [Context-Aware Sequence Alignment using 4D Skeletal Augmentation](https://mlsanity.com/paper/2204.12223/) (2022)
- [Bayesian Metric Learning for Uncertainty Quantification in Image Retrieval](https://mlsanity.com/paper/2302.01332/) (2023)
- [POV: Prompt-Oriented View-Agnostic Learning for Egocentric Hand-Object Interaction in the Multi-View World](https://mlsanity.com/paper/2403.05856/) (2024)
- [ProbVLM: Probabilistic Adapter for Frozen Vision-Language Models](https://mlsanity.com/paper/2307.00398/) (2023)
- [Introspective Deep Metric Learning](https://mlsanity.com/paper/2309.09982/) (2023)
- [Domain Generalization with Small Data](https://mlsanity.com/paper/2402.06150/) (2024)
- [Mask to reconstruct: Cooperative Semantics Completion for Video-text Retrieval](https://mlsanity.com/paper/2305.07910/) (2023)

## Abstract (arXiv)

Depictions of similar human body configurations can vary with changing viewpoints. Using only 2D information, we would like to enable vision algorithms to recognize similarity in human body poses across multiple views. This ability is useful for analyzing body movements and human behaviors in images and videos. In this paper, we propose an approach for learning a compact view-invariant embedding space from 2D joint keypoints alone, without explicitly predicting 3D poses. Since 2D poses are projected from 3D space, they have an inherent ambiguity, which is difficult to represent through a deterministic mapping. Hence, we use probabilistic embeddings to model this input uncertainty. Experimental results show that our embedding model achieves higher accuracy when retrieving similar poses across different camera views, in comparison with 2D-to-3D pose lifting models. We also demonstrate the effectiveness of applying our embeddings to view-invariant action recognition and video alignment. Our code is available at https://github.com/google-research/google-research/tree/master/poem.