# Robust Dynamic Multi-Modal Data Fusion: A Model Uncertainty Perspective

arXiv: https://arxiv.org/abs/2105.06018 · HTML: https://mlsanity.com/paper/2105.06018/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 37 references that are in the map.
- First posted on arXiv 13 May 2021.

## Builds on (most cited first, 7 of 7)

- [Dynamic Ensemble Modeling Approach to Nonstationary Neural Decoding in Brain-Computer Interfaces](https://mlsanity.com/paper/1911.00714/) (2019)
- [Particle Filtering with Invertible Particle Flow](https://mlsanity.com/paper/1607.08799/) (2016)
- [Robust Particle Filter by Dynamic Averaging of Multiple Noise Models](https://mlsanity.com/paper/1609.01336/) (2016)
- [State Space Model based Trust Evaluation over Wireless Sensor Networks: An Iterative Particle Filter Approach](https://mlsanity.com/paper/1604.00478/) (2016)
- [Online Fault-Tolerant Dynamic Event Region Detection in Sensor Networks via Trust Model](https://mlsanity.com/paper/1610.02291/) (2016)
- [ILAPF: Incremental Learning Assisted Particle Filtering](https://mlsanity.com/paper/1710.10489/) (2017)
- [A Survey on Trust Modeling from a Bayesian Perspective](https://mlsanity.com/paper/1806.03916/) (2018)

## Built on by (most cited first, 1 of 1)

- [Consistent Relative Confidence and Label-Free Model Selection for Convolutional Neural Networks](https://mlsanity.com/paper/2108.11845/) (2021)

## Abstract (arXiv)

This paper is concerned with multi-modal data fusion (MMDF) under unexpected modality failures in nonlinear non-Gaussian dynamic processes. An efficient framework to tackle this problem is proposed. In particular, a notion termed modality "usefulness", which takes a value of 1 or 0, is used for indicating whether the observation of this modality is useful or not. For n modalities involved, 2n combinations of their "usefulness" values exist. Each combination defines one hypothetical model of the true data generative process. Then the problem of concern is formalized as a task of nonlinear non-Gaussian state filtering under model uncertainty, which is addressed by a dynamic model averaging (DMA) based particle filter (PF) algorithm. This DMA algorithm employs 2n models, while all models share the same state-transition function and a unique set of particle values. That makes its computational complexity only slightly larger than a single model based PF algorithm, especially for scenarios in which n is small. Experimental results show that the proposed solution outperforms remarkably state-of-the-art methods. Code and data are available at https://github.com/robinlau1981/fusion.