# Body and Head Orientation Estimation from Low-Resolution Point Clouds in Surveillance Settings

arXiv: https://arxiv.org/abs/2311.00343 · HTML: https://mlsanity.com/paper/2311.00343/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 100 references that are in the map.
- First posted on arXiv 1 Nov 2023.

## Builds on (most cited first, 4 of 4)

- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [RealHePoNet: a robust single-stage ConvNet for head pose estimation in the wild](https://mlsanity.com/paper/2011.01890/) (2020)
- [Face-from-Depth for Head Pose Estimation on Depth Images](https://mlsanity.com/paper/1712.05277/) (2017)
- [Deep Convolutional Neural Network-based Bernoulli Heatmap for Head Pose Estimation](https://mlsanity.com/paper/2005.11780/) (2020)

## Abstract (arXiv)

We propose a system that estimates people's body and head orientations using low-resolution point cloud data from two LiDAR sensors. Our models make accurate estimations in real-world conversation settings where subjects move naturally with varying head and body poses, while seated around a table. The body orientation estimation model uses ellipse fitting while the head orientation estimation model combines geometric feature extraction with an ensemble of neural network regressors. Our models achieve a mean absolute estimation error of 5.2 degrees for body orientation and 13.7 degrees for head orientation. Compared to other body/head orientation estimation systems that use RGB cameras, our proposed system uses LiDAR sensors to preserve user privacy, while achieving comparable accuracy. Unlike other body/head orientation estimation systems, our sensors do not require a specified close-range placement in front of the subject, enabling estimation from a surveillance viewpoint which produces low-resolution data. This work is the first to attempt head orientation estimation using point clouds in a low-resolution surveillance setting. We compare our model to two state-of-the-art head orientation estimation models that are designed for high-resolution point clouds, which yield higher estimation errors on our low-resolution dataset. We also present an application of head orientation estimation by quantifying behavioral differences between neurotypical and autistic individuals in triadic (three-way) conversations. Significance tests show that autistic individuals display significantly different behavior compared to neurotypical individuals in distributing attention between conversational parties, suggesting that the approach could be a component of a behavioral analysis or coaching system.