mlsanityData to 14 Oct 2026

Multi-view Human Body Mesh Translator

arXiv 2210.01886 ↗First posted 4 Oct 2022cs.CV cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Existing methods for human mesh recovery mainly focus on single-view frameworks, but they often fail to produce accurate results due to the ill-posed setup. Considering the maturity of the multi-view motion capture system, in this paper, we propose to solve the prior ill-posed problem by leveraging multiple images from different views, thus significantly enhancing the quality of recovered meshes. In particular, we present a novel Multi-view human body Mesh Translator (MMT) model for estimating human body mesh with the help of vision transformer. Specifically, MMT takes multi-view images as input and translates them to targeted meshes in a single-forward manner. MMT fuses features of different views in both encoding and decoding phases, leading to representations embedded with global information. Additionally, to ensure the tokens are intensively focused on the human pose and shape, MMT conducts cross-view alignment at the feature level by projecting 3D keypoint positions to each view and enforcing their consistency in geometry constraints. Comprehensive experiments demonstrate that MMT outperforms existing single or multi-view models by a large margin for human mesh recovery task, notably, 28.8% improvement in MPVE over the current state-of-the-art method on the challenging HUMBI dataset. Qualitative evaluation also verifies the effectiveness of MMT in reconstructing high-quality human mesh. Codes will be made available upon acceptance.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 44 of its 56 references that are in the map. First posted on arXiv 4 Oct 2022.

Builds on

44 of 56 references.

References · 44

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  5. 2020Language Models are Few-Shot Learnerscited by 32,419
  6. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  8. 2020End-to-End Object Detection with Transformerscited by 7,414
  9. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  10. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  11. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,420
  12. 2017End-to-end Recovery of Human Shape and Posecited by 885
32 more
  1. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 651
  2. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  3. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 627
  4. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 620
  5. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 418
  6. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
  7. 2021PARE: Part Attention Regressor for 3D Human Body Estimationcited by 337
  8. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  9. 2019Convolutional Mesh Regression for Single-Image Human Shape Reconstructioncited by 270
  10. 2021Mesh Graphormercited by 265
  11. 2020Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Posecited by 248
  12. 2018Learning to Estimate 3D Human Pose and Shape from a Single Color Imagecited by 226
  13. 2020Monocular, One-stage, Regression of Multiple 3D Peoplecited by 218
  14. 2020STAR: Sparse Trained Articulated Human Body Regressorcited by 194
  15. 2018Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose and Shape Estimationcited by 185
  16. 20193D Hand Shape and Pose Estimation from a Single RGB Imagecited by 174
  17. 2019Learning the Depths of Moving People by Watching Frozen Peoplecited by 120
  18. 2019DenseRaC: Joint 3D Pose and Shape Estimation by Dense Render-and-Comparecited by 100
  19. 2017Pose-driven Deep Convolutional Model for Person Re-identificationcited by 86
  20. 2020Hierarchical Kinematic Human Mesh Recoverycited by 71
  21. 20203D Human Mesh Regression with Dense Correspondencecited by 66
  22. 2021Direct Multi-view Multi-person 3D Pose Estimationcited by 64
  23. 2021THUNDR: Transformer-based 3D HUmaN Reconstruction with Markerscited by 50
  24. 2017Pose Invariant Embedding for Deep Person Re-identificationcited by 47
  25. 2021Probabilistic 3D Human Shape and Pose Estimation from Multiple Unconstrained Images in the Wildcited by 40
  26. 2017MonoCap: Monocular Human Motion Capture using a CNN Coupled with a Geometric Priorcited by 36
  27. 2019Shape-Aware Human Pose and Shape Reconstruction Using Multi-View Imagescited by 28
  28. 2021Dynamic Multi-Person Mesh Recovery From Uncalibrated Multi-View Camerascited by 26
  29. 2020Reconstructing NBA Playerscited by 24
  30. 2018Rethinking Pose in 3D: Multi-stage Refinement and Recovery for Markerless Motion Capturecited by 17
  31. 2018Deep Autoencoder for Combined Human Pose Estimation and body Model Upscalingcited by 12
  32. 2020Detailed 3D Human Body Reconstruction from Multi-view Images Combining Voxel Super-Resolution and Learned Implicit Representationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 0; 2026: 1.

3 papers.

  1. 2024SkelFormer: Markerless 3D Pose and Shape Estimation using Skeletal Transformerscited by 2
  2. 2023BundleMoCap: Efficient, Robust and Smooth Motion Capture from Sparse Multiview Videoscited by 0
  3. 2026Monocular Models are Strong Learners for Multi-View Human Mesh Recoverycited by 0