# Omni-modal decomposition autoencoders learn full-stack wearable disentangled representations

arXiv: https://arxiv.org/abs/2608.07385 · HTML: https://mlsanity.com/paper/2608.07385/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 42 references that are in the map.
- First posted on arXiv 7 Aug 2026.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [Understanding Dimensional Collapse in Contrastive Self-supervised Learning](https://mlsanity.com/paper/2110.09348/) (2021)
- [A Cookbook of Self-Supervised Learning](https://mlsanity.com/paper/2304.12210/) (2023)
- [Measuring Disentanglement: A Review of Metrics](https://mlsanity.com/paper/2012.09276/) (2020)
- [rPPG-MAE: Self-supervised Pre-training with Masked Autoencoders for Remote Physiological Measurement](https://mlsanity.com/paper/2306.02301/) (2023)
- [Transformer-based Self-supervised Multimodal Representation Learning for Wearable Emotion Recognition](https://mlsanity.com/paper/2303.17611/) (2023)
- [Disentangled Adversarial Autoencoder for Subject-Invariant Physiological Feature Extraction](https://mlsanity.com/paper/2008.11426/) (2020)
- [Virtual Fusion with Contrastive Learning for Single Sensor-based Activity Recognition](https://mlsanity.com/paper/2312.02185/) (2023)
- [Universal Physiological Representation Learning with Soft-Disentangled Rateless Autoencoders](https://mlsanity.com/paper/2009.13453/) (2020)
- [Scaling Representation Learning from Ubiquitous ECG with State-Space Models](https://mlsanity.com/paper/2309.15292/) (2023)
- [Variational decomposition autoencoding improves disentanglement of latent representations](https://mlsanity.com/paper/2601.06844/) (2026)

## Abstract (arXiv)

Learning disentangled representations is a key requirement for developing versatile, general-purpose, and sustainable models in multi-modal wearable computing. However, existing approaches do not operate as full-stack wearable processors, i.e., they do not simultaneously address task-specific classification performance, disentangled and interpretable representation learning, fusion, and generative modeling of highly heterogeneous multi-modal time series. To address this gap, we introduce Omni-modal Variational Decomposition Autoencoders (OmniDecVAEs), a framework that efficiently learns multi-purpose representations in a unified and scalable manner from arbitrarily many modalities. OmniDecVAEs extend DecVAEs by learning modality-conditioned time-frequency latent subspaces through a multi-view self-supervised decomposition loss and a shared asymmetric autoencoder (AE) architecture. Results on a challenging omni-modal human activity recognition (HAR) setting with up to thirty modalities, demonstrate the ability of OmniDecVAEs to learn full-stack wearable representations. When compared to transformer-based and VAE-based methods, OmniDecVAEs full-stack disentangled representation properties lead to accuracy improvements of 1.01% and 6.75% in activity and identity recognition, respectively. Furthermore, OmniDecVAEs synthesize realistic omni-modal time-frequency data that manifest with enhanced reconstructions (mean absolute error improves by 76.84%) and distributional similarity between real and synthetic data (maximum mean discrepancy improves by 13.85%). Our results highlight OmniDecVAEs potential as a lightweight model suitable for intelligent edge wearables and clinical healthcare, unifying processing requirements and abilities in a single model, through its enhanced representational capacity, modality-invariant spatial complexity (4.1M parameters), and real-time latency.