# Sparse and Privacy-enhanced Representation for Human Pose Estimation

arXiv: https://arxiv.org/abs/2309.09515 · HTML: https://mlsanity.com/paper/2309.09515/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 55 references that are in the map.
- First posted on arXiv 18 Sep 2023.

## Builds on (most cited first, 30 of 38)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [OctNet: Learning Deep 3D Representations at High Resolutions](https://mlsanity.com/paper/1611.05009/) (2016)
- [Cascaded Pyramid Network for Multi-Person Pose Estimation](https://mlsanity.com/paper/1711.07319/) (2017)
- [RMPE: Regional Multi-person Pose Estimation](https://mlsanity.com/paper/1612.00137/) (2016)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [DeepPose: Human Pose Estimation via Deep Neural Networks](https://mlsanity.com/paper/1312.4659/) (2013)
- [Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1706.01307/) (2017)
- [CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark](https://mlsanity.com/paper/1812.00324/) (2018)
- [SPLATNet: Sparse Lattice Networks for Point Cloud Processing](https://mlsanity.com/paper/1802.08275/) (2018)
- [Towards Accurate Multi-person Pose Estimation in the Wild](https://mlsanity.com/paper/1701.01779/) (2017)
- [DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation](https://mlsanity.com/paper/1511.06645/) (2015)
- [DeeperCut: A Deeper, Stronger, and Faster Multi-Person Pose Estimation Model](https://mlsanity.com/paper/1605.03170/) (2016)
- [Sparse 3D convolutional neural networks](https://mlsanity.com/paper/1505.02890/) (2015)
- [Vote3Deep: Fast Object Detection in 3D Point Clouds Using Efficient Convolutional Neural Networks](https://mlsanity.com/paper/1609.06666/) (2016)
- [Learning to Anonymize Faces for Privacy Preserving Action Detection](https://mlsanity.com/paper/1803.11556/) (2018)
- [Learning Feature Pyramids for Human Pose Estimation](https://mlsanity.com/paper/1708.01101/) (2017)
- [ArcFace: Additive Angular Margin Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.07698/) (2018)
- [Towards Privacy-Preserving Visual Recognition via Adversarial Training: A Pilot Study](https://mlsanity.com/paper/1807.08379/) (2018)
- [Explicit Box Detection Unifies End-to-End Multi-Person Pose Estimation](https://mlsanity.com/paper/2302.01593/) (2023)
- [Privacy-Preserving Human Activity Recognition from Extreme Low Resolution](https://mlsanity.com/paper/1604.03196/) (2016)
- [EventHPE: Event-based 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2108.06819/) (2021)

## Built on by (most cited first, 2 of 2)

- [Exploring Event-based Human Pose Estimation with 3D Event Representations](https://mlsanity.com/paper/2311.04591/) (2023)
- [SPLite Hand: Sparsity-Aware Lightweight 3D Hand Pose Estimation](https://mlsanity.com/paper/2510.16396/) (2025)

## Abstract (arXiv)

We propose a sparse and privacy-enhanced representation for Human Pose Estimation (HPE). Given a perspective camera, we use a proprietary motion vector sensor(MVS) to extract an edge image and a two-directional motion vector image at each time frame. Both edge and motion vector images are sparse and contain much less information (i.e., enhancing human privacy). We advocate that edge information is essential for HPE, and motion vectors complement edge information during fast movements. We propose a fusion network leveraging recent advances in sparse convolution used typically for 3D voxels to efficiently process our proposed sparse representation, which achieves about 13x speed-up and 96% reduction in FLOPs. We collect an in-house edge and motion vector dataset with 16 types of actions by 40 users using the proprietary MVS. Our method outperforms individual modalities using only edge or motion vector images. Finally, we validate the privacy-enhanced quality of our sparse representation through face recognition on CelebA (a large face dataset) and a user study on our in-house dataset.