# Multimodal sensor fusion in the latent representation space

arXiv: https://arxiv.org/abs/2208.02183 · HTML: https://mlsanity.com/paper/2208.02183/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 28 references that are in the map.
- First posted on arXiv 3 Aug 2022.

## Builds on (most cited first, 10 of 10)

- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Compressed Sensing using Generative Models](https://mlsanity.com/paper/1703.03208/) (2017)
- [Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoders](https://mlsanity.com/paper/1812.01784/) (2018)
- [Multimodal Generative Models for Scalable Weakly-Supervised Learning](https://mlsanity.com/paper/1802.05335/) (2018)
- [Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models](https://mlsanity.com/paper/1911.03393/) (2019)
- [Joint Multimodal Learning with Deep Generative Models](https://mlsanity.com/paper/1611.01891/) (2016)
- [Generalized Multimodal ELBO](https://mlsanity.com/paper/2105.02470/) (2021)
- [Deep Compressed Sensing](https://mlsanity.com/paper/1905.06723/) (2019)
- [Gimme Signals: Discriminative signal encoding for multimodal activity recognition](https://mlsanity.com/paper/2003.06156/) (2020)
- [OPERAnet: A Multimodal Activity Recognition Dataset Acquired from Radio Frequency and Vision-based Sensors](https://mlsanity.com/paper/2110.04239/) (2021)

## Built on by (most cited first, 9 of 9)

- [HOP: Heterogeneous Topology-based Multimodal Entanglement for Co-Speech Gesture Generation](https://mlsanity.com/paper/2503.01175/) (2025)
- [GDTM: An Indoor Geospatial Tracking Dataset with Distributed Multimodal Sensors](https://mlsanity.com/paper/2402.14136/) (2024)
- [InfoMAE: Pair-Efficient Cross-Modal Alignment for Multimodal Time-Series Sensing Signals](https://mlsanity.com/paper/2504.09707/) (2025)
- [Streamlining Multimodal Data Fusion in Wireless Communication and Sensor Networks](https://mlsanity.com/paper/2302.12636/) (2023)
- [Non-contact Multimodal Indoor Human Monitoring Systems: A Survey](https://mlsanity.com/paper/2312.07601/) (2023)
- [Multimodal AI Systems for Enhanced Laying Hen Welfare Assessment and Productivity Optimization](https://mlsanity.com/paper/2508.07628/) (2025)
- [Towards Interpretable Visuo-Tactile Predictive Models for Soft Robot Interactions](https://mlsanity.com/paper/2407.12197/) (2024)
- [Latent Space Data Fusion Outperforms Early Fusion in Multimodal Mental Health Digital Phenotyping Data](https://mlsanity.com/paper/2507.14175/) (2025)
- [A Source-Free Approach for Domain Adaptation via Multiview Image Transformation and Latent Space Consistency](https://mlsanity.com/paper/2601.20284/) (2026)

## Abstract (arXiv)

A new method for multimodal sensor fusion is introduced. The technique relies on a two-stage process. In the first stage, a multimodal generative model is constructed from unlabelled training data. In the second stage, the generative model serves as a reconstruction prior and the search manifold for the sensor fusion tasks. The method also handles cases where observations are accessed only via subsampling i.e. compressed sensing. We demonstrate the effectiveness and excellent performance on a range of multimodal fusion experiments such as multisensory classification, denoising, and recovery from subsampled observations.