# EclipseTouch: Touch Segmentation on Ad Hoc Surfaces using Worn Infrared Shadow Casting

arXiv: https://arxiv.org/abs/2509.03430 · HTML: https://mlsanity.com/paper/2509.03430/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 77 references that are in the map.
- First posted on arXiv 3 Sep 2025.

## Builds on (most cited first, 10 of 10)

- [FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization](https://mlsanity.com/paper/2303.14189/) (2023)
- [MediaPipe Hands: On-device Real-time Hand Tracking](https://mlsanity.com/paper/2006.10214/) (2020)
- [PressureVision: Estimating Hand Pressure from a Single RGB Image](https://mlsanity.com/paper/2203.10385/) (2022)
- [EgoPressure: A Dataset for Hand Pressure and Pose Estimation in Egocentric Vision](https://mlsanity.com/paper/2409.02224/) (2024)
- [PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images](https://mlsanity.com/paper/2301.02310/) (2023)
- [Estimating Fingertip Forces, Torques, and Local Curvatures from Fingernail Images](https://mlsanity.com/paper/1909.05659/) (2019)
- [FoundHand: Large-Scale Domain-Specific Learning for Controllable Hand Image Generation](https://mlsanity.com/paper/2412.02690/) (2024)
- [EgoTouch: On-Body Touch Input Using AR/VR Headset Cameras](https://mlsanity.com/paper/2509.01786/) (2025)
- [TouchInsight: Uncertainty-aware Rapid Touch and Text Input for Mixed Reality from Egocentric Vision](https://mlsanity.com/paper/2410.05940/) (2024)
- [HaloTouch: Using IR Multi-path Interference to Support Touch Interactions With General Surfaces](https://mlsanity.com/paper/2503.01197/) (2025)

## Abstract (arXiv)

The ability to detect touch events on uninstrumented, everyday surfaces has been a long-standing goal for mixed reality systems. Prior work has shown that virtual interfaces bound to physical surfaces offer performance and ergonomic benefits over tapping at interfaces floating in the air. A wide variety of approaches have been previously developed, to which we contribute a new headset-integrated technique called systemname. We use a combination of a computer-triggered camera and one or more infrared emitters to create structured shadows, from which we can accurately estimate hover distance (mean error of 6.9~mm) and touch contact (98.0% accuracy). We discuss how our technique works across a range of conditions, including surface material, interaction orientation, and environmental lighting.