mlsanityData to 7 Oct 2026

ThermoHands: A Benchmark for 3D Hand Pose Estimation from Egocentric Thermal Images

arXiv 2403.09871 ↗First posted 14 Mar 2024cs.CV cs.AI cs.HCcited by 4

Not on the map. Why?

Abstract (arXiv)

Designing egocentric 3D hand pose estimation systems that can perform reliably in complex, real-world scenarios is crucial for downstream applications. Previous approaches using RGB or NIR imagery struggle in challenging conditions: RGB methods are susceptible to lighting variations and obstructions like handwear, while NIR techniques can be disrupted by sunlight or interference from other NIR-equipped devices. To address these limitations, we present ThermoHands, the first benchmark focused on thermal image-based egocentric 3D hand pose estimation, demonstrating the potential of thermal imaging to achieve robust performance under these conditions. The benchmark includes a multi-view and multi-spectral dataset collected from 28 subjects performing hand-object and hand-virtual interactions under diverse scenarios, accurately annotated with 3D hand poses through an automated process. We introduce a new baseline method, TherFormer, utilizing dual transformer modules for effective egocentric 3D hand pose estimation in thermal imagery. Our experimental results highlight TherFormer's leading performance and affirm thermal imaging's effectiveness in enabling robust 3D hand pose estimation in adverse conditions.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 59 of its 119 references that are in the map. First posted on arXiv 14 Mar 2024.

Builds on

59 of 119 references.

References · 59

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2023Segment Anythingcited by 8,158
  4. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  5. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
  6. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  7. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 397
  8. 2019Learning joint reconstruction of hands and manipulated objectscited by 377
  9. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 321
  10. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 320
  11. 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 306
  12. 2023Reconstructing Hands in 3D with Transformerscited by 298
38 more
  1. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 284
  2. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
  3. 2021H2O: Two Hands Manipulating Objects for First Person Interaction Recognitioncited by 251
  4. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 244
  5. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  6. 2017First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotationscited by 229
  7. 2021DexMV: Imitation Learning for Dexterous Manipulation from Human Videoscited by 194
  8. 20193D Hand Shape and Pose from Images in the Wildcited by 190
  9. 2020ContactPose: A Dataset of Grasps with Object Contact and Hand Posecited by 180
  10. 2022Embodied Hands: Modeling and Capturing Hands and Bodies Togethercited by 179
  11. 2020Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Videocited by 162
  12. 2017GANerated Hands for Real-time 3D Hand Tracking from Monocular RGBcited by 147
  13. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 140
  14. 2020Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstructioncited by 139
  15. 2022KISS-ICP: In Defense of Point-to-Point ICP -- Simple, Accurate, and Robust Registration If Done the Right Waycited by 132
  16. 2019End-to-end Hand Mesh Recovery from a Monocular RGB Imagecited by 124
  17. 2019Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Renderingcited by 115
  18. 2017V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Mapcited by 102
  19. 2020Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraintscited by 97
  20. 2022HandOccNet: Occlusion-Robust 3D Hand Mesh Estimation Networkcited by 95
  21. 2022Interacting Attention Graph for Single Image Two-Hand Reconstructioncited by 87
  22. 2017Real-time Hand Tracking under Occlusion from an Egocentric RGB-D Sensorcited by 84
  23. 2021Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Cameracited by 70
  24. 2023AssemblyHands: Towards Egocentric Activity Understanding via 3D Hand Pose Estimationcited by 64
  25. 2016Real-time Joint Tracking of a Hand Manipulating an Object from RGB-D Inputcited by 56
  26. 2019A2J: Anchor-to-Joint Regression Network for 3D Articulated Pose Estimation from a Single Depth Imagecited by 56
  27. 2017BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art Analysiscited by 46
  28. 2018Disentangling Latent Hands for Image Synthesis and Pose Estimationcited by 44
  29. 2020HeatNet: Bridging the Day-Night Domain Gap in Semantic Segmentation with Thermal Imagescited by 41
  30. 2019Segmenting Objects in Day and Night:Edge-Conditioned CNN for Thermal Image Semantic Segmentationcited by 36
  31. 2022Hierarchical Temporal Transformer for 3D Hand Pose Estimation and Action Recognition from Egocentric RGB Videoscited by 35
  32. 2020SeqHAND:RGB-Sequence-Based 3D Hand Pose and Shape Estimationcited by 34
  33. 2020Two-hand Global 3D Pose Estimation Using Monocular RGBcited by 27
  34. 2018Task-Oriented Hand Motion Retargeting for Dexterous Manipulation Imitationcited by 25
  35. 2022Pose Forecasting in Industrial Human-Robot Collaborationcited by 24
  36. 2021RGB2Hands: Real-Time Tracking of 3D Hand Interactions from Monocular RGB Videocited by 23
  37. 2019Sparse Depth Enhanced Direct Thermal-infrared SLAM Beyond the Visible Spectrumcited by 22
  38. 2023Deformer: Dynamic Fusion Transformer for Robust Hand Pose Estimationcited by 21

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 3.

4 papers.

  1. 2025Bridging Perspectives: A Survey on Cross-view Collaborative Intelligence with Egocentric-Exocentric Visioncited by 8
  2. 2024Egocentric and Exocentric Methods: A Short Surveycited by 6
  3. 2025M4Human: A Large-Scale Multimodal mmWave Radar Benchmark for Human Mesh Reconstructioncited by 6
  4. 2025THOR: Thermal-guided Hand-Object Reasoning via Adaptive Vision Samplingcited by 0