mlsanityData to 7 Oct 2026

HeatFormer: A Neural Optimizer for Multiview Human Mesh Recovery

arXiv 2412.04456 ↗First posted 5 Dec 2024cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

We introduce a novel method for human shape and pose recovery that can fully leverage multiple static views. We target fixed-multiview people monitoring, including elderly care and safety monitoring, in which calibrated cameras can be installed at the corners of a room or an open space but whose configuration may vary depending on the environment. Our key idea is to formulate it as neural optimization. We achieve this with HeatFormer, a neural optimizer that iteratively refines the SMPL parameters given multiview images, which is fundamentally agonistic to the configuration of views. HeatFormer realizes this SMPL parameter estimation as heat map generation and alignment with a novel transformer encoder and decoder. We demonstrate the effectiveness of HeatFormer including its accuracy, robustness to occlusion, and generalizability through an extensive set of experiments. We believe HeatFormer can serve a key role in passive human behavior modeling.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 48 of its 55 references that are in the map. First posted on arXiv 5 Dec 2024.

Builds on

48 of 55 references.

References · 48

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
  4. 2017End-to-end Recovery of Human Shape and Posecited by 883
  5. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
  6. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
  7. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  8. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 493
  9. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 417
  10. 2023Humans in 4D: Reconstructing and Tracking Humans with Transformerscited by 344
  11. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 319
  12. 2018Learning 3D Human Dynamics from Videocited by 315
36 more
  1. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  2. 2019Convolutional Mesh Regression for Single-Image Human Shape Reconstructioncited by 270
  3. 2021Mesh Graphormercited by 264
  4. 2021PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loopcited by 250
  5. 2020Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Posecited by 248
  6. 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 246
  7. 2018Learning to Estimate 3D Human Pose and Shape from a Single Color Imagecited by 226
  8. 2016Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Posecited by 220
  9. 2022Capturing and Inferring Dense Full-Body Human-Scene Contactcited by 176
  10. 2020Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Videocited by 162
  11. 2019Learnable Triangulation of Human Posecited by 160
  12. 2021Probabilistic Modeling for Human Mesh Recoverycited by 152
  13. 2023Decoupling Human and Camera Motion from Videos in the Wildcited by 140
  14. 2014Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimationcited by 129
  15. 2023One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformercited by 128
  16. 2024TRAM: Global Trajectory and Motion of 3D Humans from in-the-wild Videoscited by 105
  17. 2024Multi-HMR: Multi-Person Whole-Body Human Mesh Recovery in a Single Shotcited by 98
  18. 2024TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representationcited by 93
  19. 2017Towards Accurate Markerless Human Shape and Pose Estimation over Timecited by 92
  20. 2022Cross-Attention of Disentangled Modalities for 3D Human Mesh Recovery with Transformerscited by 81
  21. 2020NeuralAnnot: Neural Annotator for 3D Human Mesh Training Setscited by 62
  22. 2021Encoder-decoder with Multi-level Attention for 3D Human Shape and Pose Estimationcited by 62
  23. 2024AiOS: All-in-One-Stage Expressive Human Pose and Shape Estimationcited by 41
  24. 2020AdaFuse: Adaptive Multiview Fusion for Accurate Human Pose Estimation in the Wildcited by 40
  25. 2024Score-Guided Diffusion for 3D Human Recoverycited by 39
  26. 2023PSVT: End-to-End Multi-person 3D Pose and Shape Estimation with Progressive Video Transformerscited by 31
  27. 2023POTTER: Pooling Attention Transformer for Efficient Human Mesh Recoverycited by 31
  28. 2019Shape-Aware Human Pose and Shape Reconstruction Using Multi-View Imagescited by 28
  29. 2021Dynamic Multi-Person Mesh Recovery From Uncalibrated Multi-View Camerascited by 26
  30. 2023Co-Evolution of Pose and Mesh for 3D Human Body Estimation from Videocited by 25
  31. 2022Learnable human mesh triangulation for 3D human pose and shape estimationcited by 16
  32. 2023VQ-HPS: Human Pose and Shape Estimation in a Vector-Quantized Latent Spacecited by 13
  33. 2020Multi-view Human Pose and Shape Estimation Using Learnable Volumetric Aggregationcited by 10
  34. 2023Delving Deep into Pixel Alignment Feature for Accurate Multi-view Human Mesh Recoverycited by 10
  35. 2024MeshPose: Unifying DensePose and 3D Body Mesh reconstructioncited by 8
  36. 2024Human Mesh Recovery from Arbitrary Multi-view Imagescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026DiffProxy: Multi-View Human Mesh Recovery via Diffusion-Generated Dense Proxiescited by 1
  2. 2026Monocular Models are Strong Learners for Multi-View Human Mesh Recoverycited by 0