# Cross-view and Cross-pose Completion for 3D Human Understanding

arXiv: https://arxiv.org/abs/2311.09104 · HTML: https://mlsanity.com/paper/2311.09104/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 90 references that are in the map.
- First posted on arXiv 15 Nov 2023.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://mlsanity.com/paper/2202.03555/) (2022)
- [ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation](https://mlsanity.com/paper/2204.12484/) (2022)
- [Masked Autoencoders As Spatiotemporal Learners](https://mlsanity.com/paper/2205.09113/) (2022)
- [A Short Note on the Kinetics-700 Human Action Dataset](https://mlsanity.com/paper/1907.06987/) (2019)
- [CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completion](https://mlsanity.com/paper/2210.10716/) (2022)
- [Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases](https://mlsanity.com/paper/2007.13916/) (2020)
- [Are Large-scale Datasets Necessary for Self-Supervised Pre-training?](https://mlsanity.com/paper/2112.10740/) (2021)
- [SiT: Self-supervised vIsion Transformer](https://mlsanity.com/paper/2104.03602/) (2021)
- [Continuous Surface Embeddings](https://mlsanity.com/paper/2011.12438/) (2020)
- [The Hidden Uniform Cluster Prior in Self-Supervised Learning](https://mlsanity.com/paper/2210.07277/) (2022)
- [HaGRID - HAnd Gesture Recognition Image Dataset](https://mlsanity.com/paper/2206.08219/) (2022)
- [Generalizable Neural Performer: Learning Robust Radiance Fields for Human Novel View Synthesis](https://mlsanity.com/paper/2204.11798/) (2022)
- [Rethinking Self-Supervised Visual Representation Learning in Pre-training for 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2303.05370/) (2023)

## Built on by (most cited first, 6 of 6)

- [HAMSt3R: Human-Aware Multi-view Stereo 3D Reconstruction](https://mlsanity.com/paper/2508.16433/) (2025)
- [MEGA: Masked Generative Autoencoder for Human Mesh Recovery](https://mlsanity.com/paper/2405.18839/) (2024)
- [VQ-HPS: Human Pose and Shape Estimation in a Vector-Quantized Latent Space](https://mlsanity.com/paper/2312.08291/) (2023)
- [Human-Centric Foundation Models: Perception, Generation and Agentic Modeling](https://mlsanity.com/paper/2502.08556/) (2025)
- [Human Mesh Modeling for Anny Body](https://mlsanity.com/paper/2511.03589/) (2025)
- [EAGLE: Efficient Adaptive Geometry-based Learning in Cross-view Understanding](https://mlsanity.com/paper/2406.01429/) (2024)

## Abstract (arXiv)

Human perception and understanding is a major domain of computer vision which, like many other vision subdomains recently, stands to gain from the use of large models pre-trained on large datasets. We hypothesize that the most common pre-training strategy of relying on general purpose, object-centric image datasets such as ImageNet, is limited by an important domain shift. On the other hand, collecting domain-specific ground truth such as 2D or 3D labels does not scale well. Therefore, we propose a pre-training approach based on self-supervised learning that works on human-centric data using only images. Our method uses pairs of images of humans: the first is partially masked and the model is trained to reconstruct the masked parts given the visible ones and a second image. It relies on both stereoscopic (cross-view) pairs, and temporal (cross-pose) pairs taken from videos, in order to learn priors about 3D as well as human motion. We pre-train a model for body-centric tasks and one for hand-centric tasks. With a generic transformer architecture, these models outperform existing self-supervised pre-training methods on a wide set of human-centric downstream tasks, and obtain state-of-the-art performance for instance when fine-tuning for model-based and model-free human mesh recovery.