mlsanityData to 14 Oct 2026

Image-free Domain Generalization via CLIP for 3D Hand Pose Estimation

arXiv 2210.16788 ↗First posted 30 Oct 2022cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

RGB-based 3D hand pose estimation has been successful for decades thanks to large-scale databases and deep learning. However, the hand pose estimation network does not operate well for hand pose images whose characteristics are far different from the training data. This is caused by various factors such as illuminations, camera angles, diverse backgrounds in the input images, etc. Many existing methods tried to solve it by supplying additional large-scale unconstrained/target domain images to augment data space; however collecting such large-scale images takes a lot of labors. In this paper, we present a simple image-free domain generalization approach for the hand pose estimation framework that uses only source domain data. We try to manipulate the image features of the hand pose estimation network by adding the features from text descriptions using the CLIP (Contrastive Language-Image Pre-training) model. The manipulated image features are then exploited to train the hand pose estimation network via the contrastive learning framework. In experiments with STB and RHD datasets, our algorithm shows improved performance over the state-of-the-art domain generalization approaches.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 48 of its 65 references that are in the map. First posted on arXiv 30 Oct 2022.

Builds on

48 of 65 references.

References · 48

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  5. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  6. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  7. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  8. 2019Contrastive Multiview Codingcited by 1,301
  9. 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,082
  10. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 673
  11. 2020FDA: Fourier Domain Adaptation for Semantic Segmentationcited by 541
  12. 2019Learning joint reconstruction of hands and manipulated objectscited by 378
36 more
  1. 2019Semi-supervised Domain Adaptation via Minimax Entropycited by 335
  2. 2016Convolutional Pose Machinescited by 325
  3. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
  4. 2021Mesh Graphormercited by 265
  5. 2019Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization without Accessing Target Domain Datacited by 246
  6. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 245
  7. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  8. 2017First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotationscited by 230
  9. 2019Learning to Optimize Domain Specific Normalization for Domain Generalizationcited by 161
  10. 2019Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentationcited by 151
  11. 2017GANerated Hands for Real-time 3D Hand Tracking from Monocular RGBcited by 147
  12. 2020Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wildcited by 146
  13. 2019Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Renderingcited by 115
  14. 2020HOPE-Net: A Graph-based Model for Hand-Object Pose Estimationcited by 112
  15. 2017Adversarial Dropout Regularizationcited by 106
  16. 2020Monocular Real-time Hand Shape and Motion Capture using Multi-modal Datacited by 106
  17. 2017V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Mapcited by 102
  18. 2020Attract, Perturb, and Explore: Learning a Feature Alignment Network for Semi-supervised Domain Adaptationcited by 69
  19. 2019Self-supervised Domain Adaptation for Computer Vision Taskscited by 58
  20. 2019A2J: Anchor-to-Joint Regression Network for 3D Articulated Pose Estimation from a Single Depth Imagecited by 56
  21. 2020Two-hand Global 3D Pose Estimation Using Monocular RGBcited by 27
  22. 2020Measuring Generalisation to Unseen Viewpoints, Articulations, Shapes and Objects for 3D Hand Pose Estimation under Hand-Object Interactioncited by 24
  23. 2020HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Mapcited by 23
  24. 2020JGR-P2O: Joint Graph Reasoning based Pixel-to-Offset Prediction Network for 3D Hand Pose Estimation from a Single Depth Imagecited by 20
  25. 2017Dense 3D Regression for Hand Pose Estimationcited by 18
  26. 2020AWR: Adaptive Weighting Regression for 3D Hand Pose Estimationcited by 17
  27. 2021HandFoldingNet: A 3D Hand Pose Estimation Network Using Multiscale-Feature Guided Folding of a 2D Hand Skeletoncited by 16
  28. 2018Augmented Skeleton Space Transfer for Depth-based Hand Pose Estimationcited by 15
  29. 2017Pose Guided Structured Region Ensemble Network for Cascaded Hand Pose Estimationcited by 12
  30. 2020EqCo: Equivalent Rules for Self-supervised Contrastive Learningcited by 12
  31. 2019Sampling Strategies for GAN Synthetic Datacited by 11
  32. 2022Multi-Person 3D Pose and Shape Estimation via Inverse Kinematics and Refinementcited by 11
  33. 2019Dual Grid Net: hand mesh vertex regression from single depth mapscited by 10
  34. 2022Efficient Virtual View Selection for 3D Hand Pose Estimationcited by 6
  35. 2016Kinematic-Layout-aware Random Forests for Depth-based Action Recognitioncited by 2
  36. 2016Real-time Online Action Detection Forests using Spatio-temporal Contextscited by 2

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 0; 2025: 2.

4 papers.

  1. 2023FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learningcited by 70
  2. 2023Holistic Representation Learning for Multitask Trajectory Anomaly Detectioncited by 15
  3. 2025BIGS: Bimanual Category-agnostic Interaction Reconstruction from Monocular Videos via 3D Gaussian Splattingcited by 10
  4. 2025CoPT: Unsupervised Domain Adaptive Segmentation using Domain-Agnostic Text Embeddingscited by 7