# Fingertip Detection and Tracking for Recognition of Air-Writing in Videos

arXiv: https://arxiv.org/abs/1809.03016 · HTML: https://mlsanity.com/paper/1809.03016/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Sep 2018.

## Built on by (most cited first, 5 of 5)

- [Parallel Multi-Scale Networks with Deep Supervision for Hand Keypoint Detection](https://mlsanity.com/paper/2112.10275/) (2021)
- [Finger in Camera Speaks Everything: Unconstrained Air-Writing for Real-World](https://mlsanity.com/paper/2412.19537/) (2024)
- [Writing in The Air: Unconstrained Text Recognition from Finger Movement Using Spatio-Temporal Convolution](https://mlsanity.com/paper/2104.09021/) (2021)
- [G3R: Generating Rich and Fine-grained mmWave Radar Data from 2D Videos for Generalized Gesture Recognition](https://mlsanity.com/paper/2404.14934/) (2024)
- [Incoporating Weighted Board Learning System for Accurate Occupational Pneumoconiosis Staging](https://mlsanity.com/paper/2208.06607/) (2022)

## Abstract (arXiv)

Air-writing is the process of writing characters or words in free space using finger or hand movements without the aid of any hand-held device. In this work, we address the problem of mid-air finger writing using web-cam video as input. In spite of recent advances in object detection and tracking, accurate and robust detection and tracking of the fingertip remains a challenging task, primarily due to small dimension of the fingertip. Moreover, the initialization and termination of mid-air finger writing is also challenging due to the absence of any standard delimiting criterion. To solve these problems, we propose a new writing hand pose detection algorithm for initialization of air-writing using the Faster R-CNN framework for accurate hand detection followed by hand segmentation and finally counting the number of raised fingers based on geometrical properties of the hand. Further, we propose a robust fingertip detection and tracking approach using a new signature function called distance-weighted curvature entropy. Finally, a fingertip velocity-based termination criterion is used as a delimiter to mark the completion of the air-writing gesture. Experiments show the superiority of the proposed fingertip detection and tracking algorithm over state-of-the-art approaches giving a mean precision of 73.1 % while achieving real-time performance at 18.5 fps, a condition which is of vital importance to air-writing. Character recognition experiments give a mean accuracy of 96.11 % using the proposed air-writing system, a result which is comparable to that of existing handwritten character recognition systems.