# Coupled Support Tensor Machine Classification for Multimodal Neuroimaging Data

arXiv: https://arxiv.org/abs/2201.07683 · HTML: https://mlsanity.com/paper/2201.07683/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 82 references that are in the map.
- First posted on arXiv 19 Jan 2022.

## Builds on (most cited first, 10 of 10)

- [Tucker Tensor Regression and Neuroimaging Analysis](https://mlsanity.com/paper/1304.5637/) (2013)
- [Tensor-on-tensor regression](https://mlsanity.com/paper/1701.01037/) (2017)
- [A novel approach for fusion of heterogeneous sources of data](https://mlsanity.com/paper/1803.00138/) (2018)
- [DuSK: A Dual Structure-preserving Kernel for Supervised Tensor Learning with Applications to Neuroimages](https://mlsanity.com/paper/1407.8289/) (2014)
- [Integrative Factor Regression and Its Inference for Multimodal Data Analysis](https://mlsanity.com/paper/1911.04056/) (2019)
- [Covariate-Adjusted Tensor Classification in High-Dimensions](https://mlsanity.com/paper/1805.04421/) (2018)
- [Tensor-Based Fusion of EEG and FMRI to Understand Neurological Changes in Schizophrenia](https://mlsanity.com/paper/1612.02189/) (2016)
- [Tensor Analysis and Fusion of Multimodal Brain Images](https://mlsanity.com/paper/1506.06040/) (2015)
- [SimTensor: A synthetic tensor data generator](https://mlsanity.com/paper/1612.03772/) (2016)
- [Early soft and flexible fusion of EEG and fMRI via tensor decompositions](https://mlsanity.com/paper/2005.07134/) (2020)

## Abstract (arXiv)

Multimodal data arise in various applications where information about the same phenomenon is acquired from multiple sensors and across different imaging modalities. Learning from multimodal data is of great interest in machine learning and statistics research as this offers the possibility of capturing complementary information among modalities. Multimodal modeling helps to explain the interdependence between heterogeneous data sources, discovers new insights that may not be available from a single modality, and improves decision-making. Recently, coupled matrix-tensor factorization has been introduced for multimodal data fusion to jointly estimate latent factors and identify complex interdependence among the latent factors. However, most of the prior work on coupled matrix-tensor factors focuses on unsupervised learning and there is little work on supervised learning using the jointly estimated latent factors. This paper considers the multimodal tensor data classification problem. A Coupled Support Tensor Machine (C-STM) built upon the latent factors jointly estimated from the Advanced Coupled Matrix Tensor Factorization (ACMTF) is proposed. C-STM combines individual and shared latent factors with multiple kernels and estimates a maximal-margin classifier for coupled matrix tensor data. The classification risk of C-STM is shown to converge to the optimal Bayes risk, making it a statistically consistent rule. C-STM is validated through simulation studies as well as a simultaneous EEG-fMRI analysis. The empirical evidence shows that C-STM can utilize information from multiple sources and provide a better classification performance than traditional single-mode classifiers.