mlsanityData to 7 Oct 2026

TempCLR: Reconstructing Hands via Time-Coherent Contrastive Learning

arXiv 2209.00489 ↗First posted 1 Sep 2022cs.CVcited by 17

Not on the map. Why?

Abstract (arXiv)

We introduce TempCLR, a new time-coherent contrastive learning approach for the structured regression task of 3D hand reconstruction. Unlike previous time-contrastive methods for hand pose estimation, our framework considers temporal consistency in its augmentation scheme, and accounts for the differences of hand poses along the temporal direction. Our data-driven method leverages unlabelled videos and a standard CNN, without relying on synthetic data, pseudo-labels, or specialized architectures. Our approach improves the performance of fully-supervised hand reconstruction methods by 15.9% and 7.6% in PA-V2V on the HO-3D and FreiHAND datasets respectively, thus establishing new state-of-the-art performance. Finally, we demonstrate that our approach produces smoother hand reconstructions through time, and is more robust to heavy occlusions compared to the previous state-of-the-art which we show quantitatively and qualitatively. Our code and models will be available at https://eth-ait.github.io/tempclr.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 46 of its 49 references that are in the map. First posted on arXiv 1 Sep 2022.

Builds on

46 of 49 references.

References · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  6. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  7. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  8. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  9. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  10. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 417
  11. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 397
  12. 2019Learning joint reconstruction of hands and manipulated objectscited by 377
34 more
  1. 2020Spatiotemporal Contrastive Video Representation Learningcited by 372
  2. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 320
  3. 2020Understanding Human Hands in Contact at Internet Scalecited by 281
  4. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  5. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
  6. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  7. 2017Hand Keypoint Detection in Single Images using Multiview Bootstrappingcited by 231
  8. 2017First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotationscited by 229
  9. 20193D Hand Shape and Pose from Images in the Wildcited by 190
  10. 2020ContactPose: A Dataset of Grasps with Object Contact and Hand Posecited by 180
  11. 2022Embodied Hands: Modeling and Capturing Hands and Bodies Togethercited by 179
  12. 2020Whole-Body Human Pose Estimation in the Wildcited by 178
  13. 20193D Hand Shape and Pose Estimation from a Single RGB Imagecited by 174
  14. 2020Monocular Expressive Body Regression through Body-Driven Attentioncited by 155
  15. 2017GANerated Hands for Real-time 3D Hand Tracking from Monocular RGBcited by 147
  16. 2020Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wildcited by 146
  17. 2020Reconstructing Hand-Object Interactions in the Wildcited by 143
  18. 2021GOAL: Generating 4D Whole-Body Motion for Hand-Object Graspingcited by 142
  19. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 140
  20. 2019H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactionscited by 139
  21. 2020Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstructioncited by 139
  22. 2021ContactOpt: Optimizing Contact to Improve Graspscited by 137
  23. 2019End-to-end Hand Mesh Recovery from a Monocular RGB Imagecited by 124
  24. 2020Epipolar Transformerscited by 117
  25. 2019Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Renderingcited by 115
  26. 2018Hand Pose Estimation via Latent 2.5D Heatmap Regressioncited by 111
  27. 2021D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactionscited by 109
  28. 2020Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraintscited by 97
  29. 2018Cross-modal Deep Variational Hand Pose Estimationcited by 89
  30. 2021Learning to Disambiguate Strongly Interacting Hands via Probabilistic Per-pixel Part Segmentationcited by 40
  31. 2021Contrastive Representation Learning for Hand Shape Estimationcited by 34
  32. 2017Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goalscited by 21
  33. 2015Depth-based hand pose estimation: methods, data, and challengescited by 16
  34. 2021Adversarial Motion Modelling helps Semi-supervised Hand Pose Estimationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 4; 2025: 3; 2026: 2.

17 papers.

Sort:

By area:

  1. 2023HOLD: Category-agnostic 3D Reconstruction of Interacting Hands and Objects from Videocited by 76
  2. 2024GraspXL: Generating Grasping Motions for Diverse Objects at Scalecited by 59
  3. 2023ArtiGrasp: Physically Plausible Synthesis of Bi-Manual Dexterous Grasping and Articulationcited by 53
  4. 2024Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objectscited by 35
  5. 2023Learning to Ground Instructional Articles in Videos through Narrationscited by 28
  6. 2023HMP: Hand Motion Priors for Pose and Shape Estimation from Videocited by 23
  7. 2023Deformer: Dynamic Fusion Transformer for Robust Hand Pose Estimationcited by 21
  8. 2023Holistic Representation Learning for Multitask Trajectory Anomaly Detectioncited by 15
  9. 2025MagicHOI: Leveraging 3D Priors for Accurate Hand-object Reconstruction from Short Monocular Video Clipscited by 15
  10. 2024HandBooster: Boosting 3D Hand-Mesh Reconstruction by Conditional Synthesis and Sampling of Hand-Object Interactionscited by 13
  11. 2025SiMHand: Mining Similar Hands for Large-Scale 3D Hand Pose Pre-trainingcited by 7
  12. 2025Diffusion-based 3D Hand Motion Recovery with Intuitive Physicscited by 5
5 more
  1. 2026PAD-Hand: Physics-Aware Diffusion for Hand Motion Recoverycited by 4
  2. 20231st Place Solution of Egocentric 3D Hand Pose Estimation Challenge 2023 Technical Report:A Concise Pipeline for Egocentric Hand Pose Reconstructioncited by 3
  3. 2024Pre-Training for 3D Hand Pose Estimation with Contrastive Learning on Large-Scale Hand Images in the Wildcited by 2
  4. 2023HandMIM: Pose-Aware Self-Supervised Learning for 3D Hand Mesh Estimationcited by 0
  5. 2026Hand-4DGS: Feed-Forward 3D Gaussian Splatting for 4D Hand Reconstruction from Egocentric Videoscited by 0