# Enhanced Touchable Projector-depth System with Deep Hand Pose Estimation

arXiv: https://arxiv.org/abs/1812.11090 · HTML: https://mlsanity.com/paper/1812.11090/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Dec 2018.

## Built on by (most cited first, 1 of 1)

- [Learning to Detect Touches on Cluttered Tables](https://mlsanity.com/paper/2304.04687/) (2023)

## Abstract (arXiv)

Touchable projection with structured light range cameras is a prolific medium for large interaction surfaces, affording multiple simultaneous users and simple, cheap setup. However robust touch detection in such projector-depth systems is difficult to achieve due to measurement noise. We propose a novel combination of surface touch detection and a deep network for hand pose estimation, which aids in detecting both on- and above-surface hand gestures, disambiguating multiple touch fingers, as well as recovering fingertip positions in face of noisy input. We present the details of our GPU-accelerated system and an evaluation of its performance, as well as applications such as an enhanced virtual keyboard that utilizes the added features.