# AttX: Attentive Cross-Connections for Fusion of Wearable Signals in Emotion Recognition

arXiv: https://arxiv.org/abs/2206.04625 · HTML: https://mlsanity.com/paper/2206.04625/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 71 references that are in the map.
- First posted on arXiv 9 Jun 2022.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Self-supervised ECG Representation Learning for Emotion Recognition](https://mlsanity.com/paper/2002.03898/) (2020)
- [Self-supervised Learning for ECG-based Emotion Recognition](https://mlsanity.com/paper/1910.07497/) (2019)
- [A Transformer Architecture for Stress Detection from ECG](https://mlsanity.com/paper/2108.09737/) (2021)
- [Utilizing Deep Learning Towards Multi-modal Bio-sensing and Vision-based Affective Computing](https://mlsanity.com/paper/1905.07039/) (2019)
- [A dataset of continuous affect annotations and physiological signals for emotion analysis](https://mlsanity.com/paper/1812.02782/) (2018)
- [PARSE: Pairwise Alignment of Representations in Semi-Supervised EEG Learning for Emotion Recognition](https://mlsanity.com/paper/2202.05400/) (2022)
- [XFlow: Cross-modal Deep Neural Networks for Audiovisual Classification](https://mlsanity.com/paper/1709.00572/) (2017)
- [Unsupervised Multi-Modal Representation Learning for Affective Computing with Multi-Corpus Wearable Data](https://mlsanity.com/paper/2008.10726/) (2020)
- [Capsule Attention for Multimodal EEG-EOG Representation Learning with Application to Driver Vigilance Estimation](https://mlsanity.com/paper/1912.07812/) (2019)
- [Attentive Cross-modal Connections for Deep Multimodal Wearable-based Emotion Recognition](https://mlsanity.com/paper/2108.02241/) (2021)
- [GSR Analysis for Stress: Development and Validation of an Open Source Tool for Noisy Naturalistic GSR Data](https://mlsanity.com/paper/2005.01834/) (2020)
- [Detection of Maternal and Fetal Stress from the Electrocardiogram with Self-Supervised Representation Learning](https://mlsanity.com/paper/2011.02000/) (2020)
- [Spatio-Temporal EEG Representation Learning on Riemannian Manifold and Euclidean Space](https://mlsanity.com/paper/2008.08633/) (2020)
- [X-CNN: Cross-modal Convolutional Neural Networks for Sparse Datasets](https://mlsanity.com/paper/1610.00163/) (2016)
- [Classification of Cognitive Load and Expertise for Adaptive Simulation using Deep Multitask Learning](https://mlsanity.com/paper/1908.00385/) (2019)

## Built on by (most cited first, 6 of 6)

- [Low-Rank Adaptation of Time Series Foundational Models for Out-of-Domain Modality Forecasting](https://mlsanity.com/paper/2405.10216/) (2024)
- [Beyond LoRA: Exploring Efficient Fine-Tuning Techniques for Time Series Foundational Models](https://mlsanity.com/paper/2409.11302/) (2024)
- [CLARE: Cognitive Load Assessment in REaltime with Multimodal Data](https://mlsanity.com/paper/2404.17098/) (2024)
- [Segment, Shuffle, and Stitch: A Simple Layer for Improving Time-Series Representations](https://mlsanity.com/paper/2405.20082/) (2024)
- [Are Anxiety Detection Models Generalizable? A Cross-Activity and Cross-Population Study Using Wearables](https://mlsanity.com/paper/2504.03695/) (2025)
- [Multi-Domain EEG Representation Learning with Orthogonal Mapping and Attention-based Fusion for Cognitive Load Classification](https://mlsanity.com/paper/2511.12394/) (2025)

## Abstract (arXiv)

We propose cross-modal attentive connections, a new dynamic and effective technique for multimodal representation learning from wearable data. Our solution can be integrated into any stage of the pipeline, i.e., after any convolutional layer or block, to create intermediate connections between individual streams responsible for processing each modality. Additionally, our method benefits from two properties. First, it can share information uni-directionally (from one modality to the other) or bi-directionally. Second, it can be integrated into multiple stages at the same time to further allow network gradients to be exchanged in several touch-points. We perform extensive experiments on three public multimodal wearable datasets, WESAD, SWELL-KW, and CASE, and demonstrate that our method can effectively regulate and share information between different modalities to learn better representations. Our experiments further demonstrate that once integrated into simple CNN-based multimodal solutions (2, 3, or 4 modalities), our method can result in superior or competitive performance to state-of-the-art and outperform a variety of baseline uni-modal and classical multimodal methods.