mlsanityData to 7 Oct 2026

Rotation Equivariant 3D Hand Mesh Generation from a Single RGB Image

arXiv 2111.13023 ↗First posted 25 Nov 2021cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We develop a rotation equivariant model for generating 3D hand meshes from 2D RGB images. This guarantees that as the input image of a hand is rotated the generated mesh undergoes a corresponding rotation. Furthermore, this removes undesirable deformations in the meshes often generated by methods without rotation equivariance. By building a rotation equivariant model, through considering symmetries in the problem, we reduce the need for training on very large datasets to achieve good mesh reconstruction. The encoder takes images defined on mathbbZ2 and maps these to latent functions defined on the group C8. We introduce a novel vector mapping function to map the function defined on C8 to a latent point cloud space defined on the group SO(2). Further, we introduce a 3D projection function that learns a 3D function from the SO(2) latent space. Finally, we use an SO(3) equivariant decoder to ensure rotation equivariance. Our rotation equivariant model outperforms state-of-the-art methods on a real-world dataset and we demonstrate that it accurately captures the shape and pose in the generated meshes under rotation of the input hand.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 49 of its 60 references that are in the map. First posted on arXiv 25 Nov 2021.

Builds on

49 of 60 references.

References · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Graph Attention Networkscited by 8,854
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  5. 2018How Powerful are Graph Neural Networks?cited by 4,355
  6. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  7. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  8. 2020Point Transformercited by 1,462
  9. 2016Group Equivariant Convolutional Networkscited by 1,154
  10. 2016Geometric deep learning: going beyond Euclidean datacited by 1,042
  11. 2021E(n) Equivariant Graph Neural Networkscited by 951
  12. 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 878
37 more
  1. 2020Simple and Deep Graph Convolutional Networkscited by 857
  2. 2015Gated Graph Sequence Neural Networkscited by 792
  3. 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 751
  4. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 735
  5. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 656
  6. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 602
  7. 2017SchNet: A continuous-filter convolutional neural network for modeling quantum interactionscited by 569
  8. 2019General E(2)-Equivariant Steerable CNNscited by 441
  9. 2018Rotation Equivariant CNNs for Digital Pathologycited by 420
  10. 2019Learning joint reconstruction of hands and manipulated objectscited by 377
  11. 20183D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Datacited by 348
  12. 2017Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphscited by 307
  13. 2016Steerable CNNscited by 298
  14. 2018On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groupscited by 296
  15. 2019Gauge Equivariant Convolutional Networks and the Icosahedral CNNcited by 262
  16. 2020Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Datacited by 248
  17. 2018A General Theory of Equivariant CNNs on Homogeneous Spacescited by 223
  18. 2017Learning Steerable Filters for Rotation Equivariant CNNscited by 193
  19. 20193D Hand Shape and Pose from Images in the Wildcited by 190
  20. 2021A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groupscited by 185
  21. 20193D Hand Shape and Pose Estimation from a Single RGB Imagecited by 174
  22. 2020Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wildcited by 146
  23. 2017Equivariance Through Parameter-Sharingcited by 139
  24. 2019End-to-end Hand Mesh Recovery from a Monocular RGB Imagecited by 124
  25. 2019Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Renderingcited by 115
  26. 2018MeshNet: Mesh Neural Network for 3D Shape Representationcited by 95
  27. 2016Exploiting Cyclic Symmetry in Convolutional Neural Networkscited by 91
  28. 2018Roto-Translation Covariant Convolutional Networks for Medical Image Analysiscited by 91
  29. 2019Equivariant Flows: sampling configurations for multi-body systems with symmetric energiescited by 57
  30. 2020A Wigner-Eckart Theorem for Group Equivariant Convolution Kernelscited by 46
  31. 2020PointGMM: a Neural GMM Network for Point Cloudscited by 33
  32. 2019Single Image 3D Hand Reconstruction with Mesh Convolutionscited by 32
  33. 2018Point-to-Pose Voting based Hand Pose Estimation using Residual Permutation Equivariant Layercited by 21
  34. 2018Intertwiners between Induced Representations (with Applications to the Theory of Equivariant Neural Networks)cited by 16
  35. 2021A Graph VAE and Graph Transformer Approach to Generating Molecular Graphscited by 16
  36. 2020Rotation Equivariant Siamese Networks for Trackingcited by 15
  37. 2021Rotation Equivariant Deforestation Segmentation and Driver Classificationcited by 7

Built on by

Not cited yet.