mlsanityData to 7 Oct 2026

Sapiens: Foundation for Human Vision Models

arXiv 2408.12569 ↗First posted 22 Aug 2024cs.CVcited by 229

Not on the map. Why?

Cited by 35 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present Sapiens, a family of models for four fundamental human-centric vision tasks -- 2D pose estimation, body-part segmentation, depth estimation, and surface normal prediction. Our models natively support 1K high-resolution inference and are extremely easy to adapt for individual tasks by simply fine-tuning models pretrained on over 300 million in-the-wild human images. We observe that, given the same computational budget, self-supervised pretraining on a curated dataset of human images significantly boosts the performance for a diverse set of human-centric tasks. The resulting models exhibit remarkable generalization to in-the-wild data, even when labeled data is scarce or entirely synthetic. Our simple model design also brings scalability -- model performance across tasks improves as we scale the number of parameters from 0.3 to 2 billion. Sapiens consistently surpasses existing baselines across various human-centric benchmarks. We achieve significant improvements over the prior state-of-the-art on Humans-5K (pose) by 7.6 mAP, Humans-2K (part-seg) by 17.1 mIoU, Hi4D (depth) by 22.4% relative RMSE, and THuman2 (normal) by 53.5% relative angular error. Project page: https://about.meta.com/realitylabs/codecavatars/sapiens.

Key facts. Cited by 229 papers in the map (data through 7 Oct 2026). Builds on 99 of its 121 references that are in the map. First posted on arXiv 22 Aug 2024.

Builds on

99 of 121 references.

References · 99

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2023GPT-4 Technical Reportcited by 18,132
  8. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  9. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  10. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  11. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  12. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
38 more
  1. 2023Segment Anythingcited by 8,158
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2022Scalable Diffusion Models with Transformerscited by 5,931
  8. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  9. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  10. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  12. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  13. 2023Mistral 7Bcited by 3,065
  14. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  15. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  16. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  17. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  18. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  19. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  20. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  21. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  22. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  23. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  24. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  25. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
  26. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  27. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  28. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
  29. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  30. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  31. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  32. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 690
  33. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  34. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 683
  35. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
  36. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  37. 2020AdaBins: Depth Estimation using Adaptive Binscited by 596
  38. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 566

50 most cited of 99 All 99 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 129; 2026: 86.

229 papers.

Sort:

By area:

  1. 2025SkyReels-A2: Compose Anything in Video Diffusion Transformerscited by 72
  2. 2025PromptHMR: Promptable Human Mesh Recoverycited by 70
  3. 2025LHM: Large Animatable Human Reconstruction Model from a Single Image in Secondscited by 61
  4. 2025Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordancecited by 60
  5. 2024IDOL: Instant Photorealistic 3D Human Creation from a Single Imagecited by 55
  6. 2025Avat3r: Large Animatable Gaussian Reconstruction Model for High-fidelity 3D Head Avatarscited by 52
  7. 2025HumanDiT: Pose-Guided Diffusion Transformer for Long-form Human Motion Video Generationcited by 38
  8. 2024AniGS: Animatable Gaussian Avatar from a Single Image with Inconsistent Gaussian Reconstructioncited by 36
  9. 2025SMPLest-X: Ultimate Scaling for Expressive Human Pose and Shape Estimationcited by 31
  10. 2025Vid2Avatar-Pro: Authentic Avatar from Videos in the Wild via Universal Priorcited by 31
  11. 2025Uni4D: Unifying Visual Foundation Models for 4D Modeling from a Single Videocited by 27
  12. 2024Feat2GS: Probing Visual Foundation Models with Gaussian Splattingcited by 26
38 more
  1. 2025Generative Physical AI in Vision: A Surveycited by 24
  2. 2024PSHuman: Photorealistic Single-image 3D Human Reconstruction using Cross-Scale Multiview Diffusion and Explicit Remeshingcited by 23
  3. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 23
  4. 2025VidCapBench: A Comprehensive Benchmark of Video Captioning for Controllable Text-to-Video Generationcited by 22
  5. 2025Pippo: High-Resolution Multi-View Humans from a Single Imagecited by 20
  6. 2025Referring to Any Personcited by 20
  7. 2025Pixel3DMM: Versatile Screen-Space Priors for Single-Image 3D Face Reconstructioncited by 20
  8. 2025URDF-Anything: Constructing Articulated Objects with 3D Multimodal Language Modelcited by 20
  9. 2025EgoExo-Gen: Ego-centric Video Prediction by Watching Exo-centric Videoscited by 19
  10. 2025InsTaG: Learning Personalized 3D Talking Head from Few-Second Videocited by 18
  11. 2025Diffuman4D: 4D Consistent Human View Synthesis from Sparse-View Videos with Spatio-Temporal Diffusion Modelscited by 17
  12. 2025Embody 3D: A Large-scale Multimodal Motion and Behavior Datasetcited by 16
  13. 2024PERSE: Personalized 3D Generative Avatars from A Single Portraitcited by 15
  14. 2025Synthetic Video Enhances Physical Fidelity in Video Synthesiscited by 15
  15. 2025Precise Action-to-Video Generation Through Visual Action Promptscited by 15
  16. 2025SapiensID: Foundation for Human Recognitioncited by 14
  17. 2025REWIND: Real-Time Egocentric Whole-Body Motion Diffusion with Exemplar-Based Identity Conditioningcited by 14
  18. 2024Second FRCSyn-onGoing: Winning Solutions and Post-Challenge Analysis to Improve Face Recognition with Synthetic Datacited by 13
  19. 2025Multi-identity Human Image Animation with Structural Video Diffusioncited by 13
  20. 2025CoMotion: Concurrent Multi-person 3D Motioncited by 13
  21. 2025AGHI-QA: A Subjective-Aligned Dataset and Metric for AI-Generated Human Imagescited by 13
  22. 2025X-UniMotion: Animating Human Images with Expressive, Unified and Identity-Agnostic Motion Latentscited by 13
  23. 2025PERSONA: Personalized Whole-Body 3D Avatar with Pose-Driven Deformations from a Single Imagecited by 13
  24. 2025SIGMAN:Scaling 3D Human Gaussian Generation with Millions of Assetscited by 12
  25. 2025Single View Garment Reconstruction Using Diffusion Mapping Via Pattern Coordinatescited by 12
  26. 2025DreamCube: 3D Panorama Generation via Multi-plane Synchronizationcited by 12
  27. 2025ATLAS: Decoupling Skeletal and Shape Parameters for Expressive Parametric Human Modelingcited by 12
  28. 2026Video Generation Models are General-Purpose Vision Learnerscited by 12
  29. 2025DAViD: Data-efficient and Accurate Vision Models from Synthetic Datacited by 11
  30. 2024SqueezeMe: Mobile-Ready Distillation of Gaussian Full-Body Avatarscited by 10
  31. 2025Human-Centric Foundation Models: Perception, Generation and Agentic Modelingcited by 10
  32. 2025MMGen: Unified Multi-modal Image Generation and Understanding in One Gocited by 10
  33. 2025Monocular and Generalizable Gaussian Talking Head Animationcited by 10
  34. 2025MVHumanNet++: A Large-scale Dataset of Multi-view Daily Dressing Human Captures with Richer Annotations for 3D Human Digitizationcited by 9
  35. 2025InfinityHuman: Towards Long-Term Audio-Driven Humancited by 9
  36. 2024FoundHand: Large-Scale Domain-Specific Learning for Controllable Hand Image Generationcited by 8
  37. 2025UniGaze: Towards Universal Gaze Estimation via Large-scale Pre-Trainingcited by 8
  38. 2025MoGA: 3D Generative Avatar Prior for Monocular Gaussian Avatar Reconstructioncited by 8

50 most cited of 229 All 229 (JSON)