# ACR: Attention Collaboration-based Regressor for Arbitrary Two-Hand Reconstruction

arXiv: https://arxiv.org/abs/2303.05938 · HTML: https://mlsanity.com/paper/2303.05938/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 45 references that are in the map.
- First posted on arXiv 10 Mar 2023.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [End-to-End Human Pose and Mesh Reconstruction with Transformers](https://mlsanity.com/paper/2012.09760/) (2020)
- [PARE: Part Attention Regressor for 3D Human Body Estimation](https://mlsanity.com/paper/2104.08527/) (2021)
- [An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution](https://mlsanity.com/paper/1807.03247/) (2018)
- [I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Image](https://mlsanity.com/paper/2008.03713/) (2020)
- [FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Images](https://mlsanity.com/paper/1909.04349/) (2019)
- [Mesh Graphormer](https://mlsanity.com/paper/2104.00272/) (2021)
- [HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation](https://mlsanity.com/paper/1908.10357/) (2019)
- [InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/2008.09309/) (2020)
- [Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies](https://mlsanity.com/paper/1801.01615/) (2018)
- [Learning to Estimate 3D Hand Pose from Single RGB Images](https://mlsanity.com/paper/1705.01389/) (2017)
- [Monocular, One-stage, Regression of Multiple 3D People](https://mlsanity.com/paper/2008.12272/) (2020)
- [3D Hand Shape and Pose from Images in the Wild](https://mlsanity.com/paper/1902.03451/) (2019)
- [Embodied Hands: Modeling and Capturing Hands and Bodies Together](https://mlsanity.com/paper/2201.02610/) (2022)
- [3D Hand Shape and Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/1903.00812/) (2019)
- [Monocular Total Capture: Posing Face, Body, and Hands in the Wild](https://mlsanity.com/paper/1812.01598/) (2018)
- [Monocular Expressive Body Regression through Body-Driven Attention](https://mlsanity.com/paper/2008.09062/) (2020)
- [Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild](https://mlsanity.com/paper/2004.01946/) (2020)
- [Collaborative Regression of Expressive Bodies using Moderation](https://mlsanity.com/paper/2105.05301/) (2021)
- [End-to-end Hand Mesh Recovery from a Monocular RGB Image](https://mlsanity.com/paper/1902.09305/) (2019)
- [Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimation](https://mlsanity.com/paper/2104.14639/) (2021)
- [Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Rendering](https://mlsanity.com/paper/1904.04196/) (2019)
- [Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data](https://mlsanity.com/paper/2003.09572/) (2020)
- [Cross-modal Deep Variational Hand Pose Estimation](https://mlsanity.com/paper/1803.11404/) (2018)
- [Interacting Attention Graph for Single Image Two-Hand Reconstruction](https://mlsanity.com/paper/2203.09364/) (2022)
- [Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Camera](https://mlsanity.com/paper/2106.08059/) (2021)
- [Monocular Real-time Full Body Capture with Inter-part Correlations](https://mlsanity.com/paper/2012.06087/) (2020)

## Built on by (most cited first, 30 of 37)

- [Reconstructing Hands in 3D with Transformers](https://mlsanity.com/paper/2312.05251/) (2023)
- [Dyn-HaMR: Recovering 4D Interacting Hand Motion from a Dynamic Camera](https://mlsanity.com/paper/2412.12861/) (2024)
- [SignAvatars: A Large-scale 3D Sign Language Holistic Motion Dataset and Benchmark](https://mlsanity.com/paper/2310.20436/) (2023)
- [InterHandGen: Two-Hand Interaction Generation via Cascaded Reverse Diffusion](https://mlsanity.com/paper/2403.17422/) (2024)
- [Reconstructing Hand-Held Objects in 3D from Images and Videos](https://mlsanity.com/paper/2404.06507/) (2024)
- [FourierHandFlow: Neural 4D Hand Representation Using Fourier Query Flow](https://mlsanity.com/paper/2307.08100/) (2023)
- [DICE: End-to-end Deformation Capture of Hand-Face Interactions from a Single Image](https://mlsanity.com/paper/2406.17988/) (2024)
- [HandOS: 3D Hand Reconstruction in One Stage](https://mlsanity.com/paper/2412.01537/) (2024)
- [EgoWorld: Translating Exocentric View to Egocentric View using Rich Exocentric Observations](https://mlsanity.com/paper/2506.17896/) (2025)
- [BiTT: Bi-directional Texture Reconstruction of Interacting Two Hands from a Single Image](https://mlsanity.com/paper/2403.08262/) (2024)
- [GraspDiffusion: Synthesizing Realistic Whole-body Hand-Object Interaction](https://mlsanity.com/paper/2410.13911/) (2024)
- [Perceiving and Acting in First-Person: A Dataset and Benchmark for Egocentric Human-Object-Human Interactions](https://mlsanity.com/paper/2508.04681/) (2025)
- [Ins-HOI: Instance Aware Human-Object Interactions Recovery](https://mlsanity.com/paper/2312.09641/) (2023)
- [EgoGrasp: World-Space Hand-Object Interaction Estimation from Egocentric Videos](https://mlsanity.com/paper/2601.01050/) (2026)
- [HuTuMotion: Human-Tuned Navigation of Latent Motion Diffusion Models with Minimal Feedback](https://mlsanity.com/paper/2312.12227/) (2023)
- [CLIP-Hand3D: Exploiting 3D Hand Pose Estimation via Context-Aware Prompting](https://mlsanity.com/paper/2309.16140/) (2023)
- [Weakly-Supervised 3D Hand Reconstruction with Knowledge Prior and Uncertainty Guidance](https://mlsanity.com/paper/2407.12307/) (2024)
- [Diffusion-based 3D Hand Motion Recovery with Intuitive Physics](https://mlsanity.com/paper/2508.01835/) (2025)
- [Text-Driven 3D Hand Motion Generation from Sign Language Data](https://mlsanity.com/paper/2508.15902/) (2025)
- [PAD-Hand: Physics-Aware Diffusion for Hand Motion Recovery](https://mlsanity.com/paper/2603.26068/) (2026)
- [Generative Hierarchical Temporal Transformer for Hand Pose and Action Modeling](https://mlsanity.com/paper/2311.17366/) (2023)
- [Learning Interaction-aware 3D Gaussian Splatting for One-shot Hand Avatars](https://mlsanity.com/paper/2410.08840/) (2024)
- [Posture-Informed Muscular Force Learning for Robust Hand Pressure Estimation](https://mlsanity.com/paper/2410.23629/) (2024)
- [Towards Dynamic 3D Reconstruction of Hand-Instrument Interaction in Ophthalmic Surgery](https://mlsanity.com/paper/2505.17677/) (2025)
- [Pyramid Deep Fusion Network for Two-Hand Reconstruction from RGB-D Images](https://mlsanity.com/paper/2307.06038/) (2023)
- [Denoising Diffusion for 3D Hand Pose Estimation from Images](https://mlsanity.com/paper/2308.09523/) (2023)
- [VM-BHINet:Vision Mamba Bimanual Hand Interaction Network for 3D Interacting Hand Mesh Recovery From a Single RGB Image](https://mlsanity.com/paper/2504.14618/) (2025)
- [Two Hands Are Better Than One: Resolving Hand to Hand Intersections via Occupancy Networks](https://mlsanity.com/paper/2404.05414/) (2024)
- [Quality assessment of 3D human animation: Subjective and objective evaluation](https://mlsanity.com/paper/2505.23301/) (2025)
- [DexAvatar: 3D Sign Language Reconstruction with Hand and Body Pose Priors](https://mlsanity.com/paper/2512.21054/) (2025)

## Abstract (arXiv)

Reconstructing two hands from monocular RGB images is challenging due to frequent occlusion and mutual confusion. Existing methods mainly learn an entangled representation to encode two interacting hands, which are incredibly fragile to impaired interaction, such as truncated hands, separate hands, or external occlusion. This paper presents ACR (Attention Collaboration-based Regressor), which makes the first attempt to reconstruct hands in arbitrary scenarios. To achieve this, ACR explicitly mitigates interdependencies between hands and between parts by leveraging center and part-based attention for feature extraction. However, reducing interdependence helps release the input constraint while weakening the mutual reasoning about reconstructing the interacting hands. Thus, based on center attention, ACR also learns cross-hand prior that handle the interacting hands better. We evaluate our method on various types of hand reconstruction datasets. Our method significantly outperforms the best interacting-hand approaches on the InterHand2.6M dataset while yielding comparable performance with the state-of-the-art single-hand methods on the FreiHand dataset. More qualitative results on in-the-wild and hand-object interaction datasets and web images/videos further demonstrate the effectiveness of our approach for arbitrary hand reconstruction. Our code is available at https://github.com/ZhengdiYu/Arbitrary-Hands-3D-Reconstruction.