# Physical Self-Supervised Learning: IMU Sensing without Manual Labels

arXiv: https://arxiv.org/abs/2607.18361 · HTML: https://mlsanity.com/paper/2607.18361/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 90 references that are in the map.
- First posted on arXiv 20 Jul 2026.

## Builds on (most cited first, 30 of 32)

- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [Spherical CNNs](https://mlsanity.com/paper/1801.10130/) (2018)
- [Foundation Models for Time Series Analysis: A Tutorial and Survey](https://mlsanity.com/paper/2403.14735/) (2024)
- [Physical Inertial Poser (PIP): Physics-aware Real-time Human Motion Tracking from Sparse Inertial Sensors](https://mlsanity.com/paper/2203.08528/) (2022)
- [Deep Learning for Time Series Forecasting: Tutorial and Literature Survey](https://mlsanity.com/paper/2004.10240/) (2020)
- [Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUs](https://mlsanity.com/paper/1703.08014/) (2017)
- [DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing](https://mlsanity.com/paper/1611.01942/) (2016)
- [RoNIN: Robust Neural Inertial Navigation in the Wild: Benchmark, Evaluations, and New Methods](https://mlsanity.com/paper/1905.12853/) (2019)
- [IONet: Learning to Cure the Curse of Drift in Inertial Odometry](https://mlsanity.com/paper/1802.02209/) (2018)
- [TLIO: Tight Learned Inertial Odometry](https://mlsanity.com/paper/2007.01867/) (2020)
- [Contrastive Predictive Coding for Human Activity Recognition](https://mlsanity.com/paper/2012.05333/) (2020)
- [Deep Inertial Poser: Learning to Reconstruct Human Pose from Sparse Inertial Measurements in Real Time](https://mlsanity.com/paper/1810.04703/) (2018)
- [What Makes Good Contrastive Learning on Small-Scale Wearable-based Tasks?](https://mlsanity.com/paper/2202.05998/) (2022)
- [Dynamic Inertial Poser (DynaIP): Part-Based Motion Dynamics Learning for Enhanced Human Pose Estimation with Sparse Inertial Sensors](https://mlsanity.com/paper/2312.02196/) (2023)
- [TransPose: Real-time 3D Human Translation and Pose Estimation with Six Inertial Sensors](https://mlsanity.com/paper/2105.04605/) (2021)
- [OxIOD: The Dataset for Deep Inertial Odometry](https://mlsanity.com/paper/1809.07491/) (2018)
- [DeepIoT: Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Framework](https://mlsanity.com/paper/1706.01215/) (2017)
- [IMUTube: Automatic Extraction of Virtual on-body Accelerometry from Video for Human Activity Recognition](https://mlsanity.com/paper/2006.05675/) (2020)
- [SparsePoser: Real-time Full-body Motion Reconstruction from Sparse Data](https://mlsanity.com/paper/2311.02191/) (2023)
- [Deep Learning-Based Gait Recognition Using Smartphones in the Wild](https://mlsanity.com/paper/1811.00338/) (2018)
- [Generating Virtual On-body Accelerometer Data from Virtual Textual Descriptions for Human Activity Recognition](https://mlsanity.com/paper/2305.03187/) (2023)
- [FastDeepIoT: Towards Understanding and Optimizing Neural Network Execution Time on Mobile and Embedded Devices](https://mlsanity.com/paper/1809.06970/) (2018)
- [CTIN: Robust Contextual Transformer Network for Inertial Navigation](https://mlsanity.com/paper/2112.02143/) (2021)
- [PhyAug: Physics-Directed Data Augmentation for Deep Sensing Model Transfer in Cyber-Physical Systems](https://mlsanity.com/paper/2104.01160/) (2021)
- [Self-supervised Human Activity Recognition by Learning to Predict Cross-Dimensional Motion](https://mlsanity.com/paper/2010.13713/) (2020)

## Built on by (most cited first, 2 of 2)

- [LeanStream: A Speculate-and-Refine Streaming Framework for Efficient on-Device LLM Inference](https://mlsanity.com/paper/2609.03079/) (2026)
- [IMU-Bench: Benchmarking Wrist IMU Representations Across Kinematic, Semantic and Clinical Tasks](https://mlsanity.com/paper/2610.19481/) (2026)

## Abstract (arXiv)

Deep neural networks have become a promising approach for IMU-based sensing, but their scalability is fundamentally limited by costly labeled data and poor robustness to heterogeneous devices, placements, and users. Existing unsupervised and self-supervised methods reduce but do not remove this dependence, still requiring labeled data for domain adaptation and largely ignoring known physical structure. We propose physical self-supervised learning, an autoencoder-style paradigm for label-free IMU sensing. We replace the conventional neural decoder with an auto-adaptive physics decoder, a learnable family of kinematic equations that enforces explicit physical structure while adapting across environments, and adopt a hybrid two-stage IMU encoder with reconstruction in a structured latent space to mitigate sensor noise. Our framework further introduces probabilistic frequency-spatial constraints to disentangle sensor and object motion, a multi-view kinematic tree to exploit sparse physical self-supervised signals, and an uncertainty-aware formulation to handle the inherent ambiguity of IMU inference. Evaluated on inertial tracking and full-body motion capture over public datasets and realistic deployments, physical self-supervised learning reduces errors by up to 5x for tracking and 4x for motion capture in challenging generalization scenarios, consistently outperforming state-of-the-art supervised and self-supervised baselines without any labels. Our code is available at https://github.com/YuyangLeng/physical-ssl-imu-label-free