# CROMOSim: A Deep Learning-based Cross-modality Inertial Measurement Simulator

arXiv: https://arxiv.org/abs/2202.10562 · HTML: https://mlsanity.com/paper/2202.10562/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 27 references that are in the map.
- First posted on arXiv 21 Feb 2022.

## Builds on (most cited first, 9 of 9)

- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [VIBE: Video Inference for Human Body Pose and Shape Estimation](https://mlsanity.com/paper/1912.05656/) (2019)
- [Consistent Video Depth Estimation](https://mlsanity.com/paper/2004.15021/) (2020)
- [Robust Consistent Video Depth Estimation](https://mlsanity.com/paper/2012.05901/) (2020)
- [Deep Learning for Sensor-based Activity Recognition: A Survey](https://mlsanity.com/paper/1707.03502/) (2017)
- [Recognizing Detailed Human Context In-the-Wild from Smartphones and Smartwatches](https://mlsanity.com/paper/1609.06354/) (2016)
- [A Deep Learning Method for Complex Human Activity Recognition Using Virtual Wearable Sensors](https://mlsanity.com/paper/2003.01874/) (2020)
- [Yet it moves: Learning from Generic Motions to Generate IMU data from YouTube videos](https://mlsanity.com/paper/2011.11600/) (2020)

## Built on by (most cited first, 6 of 6)

- [Past, Present, and Future of Sensor-Based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task](https://mlsanity.com/paper/2411.14452/) (2024)
- [A Survey on Multimodal Wearable Sensor-based Human Action Recognition](https://mlsanity.com/paper/2404.15349/) (2024)
- [A Survey of IMU Based Cross-Modal Transfer Learning in Human Activity Recognition](https://mlsanity.com/paper/2403.15444/) (2024)
- [C3T: Cross-modal Transfer Through Time for Sensor-based Human Activity Recognition](https://mlsanity.com/paper/2407.16803/) (2024)
- [SImpHAR: Advancing impedance-based human activity recognition using 3D simulation and text-to-motion models](https://mlsanity.com/paper/2507.06405/) (2025)
- [Understanding the Representation of Older Adults in Motion Capture Locomotion Datasets](https://mlsanity.com/paper/2511.11713/) (2025)

## Abstract (arXiv)

With the prevalence of wearable devices, inertial measurement unit (IMU) data has been utilized in monitoring and assessment of human mobility such as human activity recognition (HAR). Training deep neural network (DNN) models for these tasks require a large amount of labeled data, which are hard to acquire in uncontrolled environments. To mitigate the data scarcity problem, we design CROMOSim, a cross-modality sensor simulator that simulates high fidelity virtual IMU sensor data from motion capture systems or monocular RGB cameras. It utilizes a skinned multi-person linear model (SMPL) for 3D body pose and shape representations, to enable simulation from arbitrary on-body positions. A DNN model is trained to learn the functional mapping from imperfect trajectory estimations in a 3D SMPL body tri-mesh due to measurement noise, calibration errors, occlusion and other modeling artifacts, to IMU data. We evaluate the fidelity of CROMOSim simulated data and its utility in data augmentation on various HAR datasets. Extensive experiment results show that the proposed model achieves a 6.7% improvement over baseline methods in a HAR task.