# Integrated In-vehicle Monitoring System Using 3D Human Pose Estimation and Seat Belt Segmentation

arXiv: https://arxiv.org/abs/2204.07946 · HTML: https://mlsanity.com/paper/2204.07946/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 32 references that are in the map.
- First posted on arXiv 17 Apr 2022.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges](https://mlsanity.com/paper/1902.07830/) (2019)
- [Integral Human Pose Regression](https://mlsanity.com/paper/1711.08229/) (2017)
- [Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose](https://mlsanity.com/paper/1611.07828/) (2016)
- [Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB](https://mlsanity.com/paper/1712.03453/) (2017)
- [Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/1907.11346/) (2019)
- [Compositional Human Pose Regression](https://mlsanity.com/paper/1704.00159/) (2017)
- [PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation](https://mlsanity.com/paper/2105.02465/) (2021)
- [XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera](https://mlsanity.com/paper/1907.00837/) (2019)
- [HMOR: Hierarchical Multi-Person Ordinal Relations for Monocular Multi-Person 3D Pose Estimation](https://mlsanity.com/paper/2008.00206/) (2020)
- [Compressed Volumetric Heatmaps for Multi-Person 3D Pose Estimation](https://mlsanity.com/paper/2004.00329/) (2020)
- [Importance-Aware Semantic Segmentation in Self-Driving with Discrete Wasserstein Training](https://mlsanity.com/paper/2010.12440/) (2020)
- [HDNet: Human Depth Estimation for Multi-Person Camera-Space Localization](https://mlsanity.com/paper/2007.08943/) (2020)
- [Looking at Hands in Autonomous Vehicles: A ConvNet Approach using Part Affinity Fields](https://mlsanity.com/paper/1804.01176/) (2018)
- [LiftFormer: 3D Human Pose Estimation using attention models](https://mlsanity.com/paper/2009.00348/) (2020)
- [NADS-Net: A Nimble Architecture for Driver and Seat Belt Detection via Convolutional Neural Networks](https://mlsanity.com/paper/1910.03695/) (2019)

## Abstract (arXiv)

Recently, along with interest in autonomous vehicles, the importance of monitoring systems for both drivers and passengers inside vehicles has been increasing. This paper proposes a novel in-vehicle monitoring system the combines 3D pose estimation, seat-belt segmentation, and seat-belt status classification networks. Our system outputs various information necessary for monitoring by accurately considering the data characteristics of the in-vehicle environment. Specifically, the proposed 3D pose estimation directly estimates the absolute coordinates of keypoints for a driver and passengers, and the proposed seat-belt segmentation is implemented by applying a structure based on the feature pyramid. In addition, we propose a classification task to distinguish between normal and abnormal states of wearing a seat belt using results that combine 3D pose estimation with seat-belt segmentation. These tasks can be learned simultaneously and operate in real-time. Our method was evaluated on a private dataset we newly created and annotated. The experimental results show that our method has significantly high performance that can be applied directly to real in-vehicle monitoring systems.