# EmbraceNet: A robust deep learning architecture for multimodal classification

arXiv: https://arxiv.org/abs/1904.09078 · HTML: https://mlsanity.com/paper/1904.09078/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Apr 2019.

## Built on by (most cited first, 30 of 30)

- [Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptation](https://mlsanity.com/paper/2310.03986/) (2023)
- [COSMOS: Catching Out-of-Context Misinformation with Self-Supervised Learning](https://mlsanity.com/paper/2101.06278/) (2021)
- [Imbalanced Big Data Oversampling: Taxonomy, Algorithms, Software, Guidelines and Future Directions](https://mlsanity.com/paper/2107.11508/) (2021)
- [Learning Multimodal Data Augmentation in Feature Space](https://mlsanity.com/paper/2212.14453/) (2022)
- [SFusion: Self-attention based N-to-One Multimodal Fusion Block](https://mlsanity.com/paper/2208.12776/) (2022)
- [Human Activity Recognition Using Tools of Convolutional Neural Networks: A State of the Art Review, Data Sets, Challenges and Future Prospects](https://mlsanity.com/paper/2202.03274/) (2022)
- [MMP: Towards Robust Multi-Modal Learning with Masked Modality Projection](https://mlsanity.com/paper/2410.03010/) (2024)
- [Do Images really do the Talking? Analysing the significance of Images in Tamil Troll meme classification](https://mlsanity.com/paper/2108.03886/) (2021)
- [MST-Distill: Mixture of Specialized Teachers for Cross-Modal Knowledge Distillation](https://mlsanity.com/paper/2507.07015/) (2025)
- [Neural Sequence-to-Sequence Modeling with Attention by Leveraging Deep Learning Architectures for Enhanced Contextual Understanding in Abstractive Text Summarization](https://mlsanity.com/paper/2404.08685/) (2024)
- [Completed Feature Disentanglement Learning for Multimodal MRIs Analysis](https://mlsanity.com/paper/2407.04916/) (2024)
- [GLAMI-1M: A Multilingual Image-Text Fashion Dataset](https://mlsanity.com/paper/2211.14451/) (2022)
- [Whose Emotion Matters? Speaking Activity Localisation without Prior Knowledge](https://mlsanity.com/paper/2211.15377/) (2022)
- [Multimodal Representation Learning and Fusion](https://mlsanity.com/paper/2506.20494/) (2025)
- [Modular Multimodal Machine Learning for Extraction of Theorems and Proofs in Long Scientific Documents (Extended Version)](https://mlsanity.com/paper/2307.09047/) (2023)
- [Missing Data as Augmentation in the Earth Observation Domain: A Multi-View Learning Approach](https://mlsanity.com/paper/2501.01132/) (2025)
- [Robust Multimodal Learning via Cross-Modal Proxy Tokens](https://mlsanity.com/paper/2501.17823/) (2025)
- [Multi-modal Co-learning for Earth Observation: Enhancing single-modality models via modality collaboration](https://mlsanity.com/paper/2510.19579/) (2025)
- [Facing the Void: Overcoming Missing Data in Multi-View Imagery](https://mlsanity.com/paper/2205.10592/) (2022)
- [Using Scene and Semantic Features for Multi-modal Emotion Recognition](https://mlsanity.com/paper/2308.00228/) (2023)
- [GAME: Generalized deep learning model towards multimodal data integration for early screening of adolescent mental disorders](https://mlsanity.com/paper/2309.10077/) (2023)
- [Conformal Calibration for Multi-Modal Regression with Missing Modalities](https://mlsanity.com/paper/2608.07795/) (2026)
- [A Light-weight Deep Human Activity Recognition Algorithm Using Multi-knowledge Distillation](https://mlsanity.com/paper/2107.07331/) (2021)
- [Multimodal Classification: Current Landscape, Taxonomy and Future Directions](https://mlsanity.com/paper/2109.09020/) (2021)
- [Real Estate Attribute Prediction from Multiple Visual Modalities with Missing Data](https://mlsanity.com/paper/2211.09018/) (2022)
- [A Contrastive Variational AutoEncoder for NSCLC Survival Prediction with Missing Modalities](https://mlsanity.com/paper/2602.17402/) (2026)
- [Evidential Fusion Network for Multimodal Survival Prediction under Missing Modalities](https://mlsanity.com/paper/2606.20757/) (2026)
- [Co-Learning for Missing Arbitrary Modalities in Multi-modal Classification](https://mlsanity.com/paper/2607.24683/) (2026)
- [Self-Evolving Multimedia Verification through Memory Consolidation of Contestation Experiences](https://mlsanity.com/paper/2609.27175/) (2026)
- [TSMD: Temporal-Stream Modality Dropout for Robust Video Highlight Detection](https://mlsanity.com/paper/2609.39051/) (2026)

## Abstract (arXiv)

Classification using multimodal data arises in many machine learning applications. It is crucial not only to model cross-modal relationship effectively but also to ensure robustness against loss of part of data or modalities. In this paper, we propose a novel deep learning-based multimodal fusion architecture for classification tasks, which guarantees compatibility with any kind of learning models, deals with cross-modal information carefully, and prevents performance degradation due to partial absence of data. We employ two datasets for multimodal classification tasks, build models based on our architecture and other state-of-the-art models, and analyze their performance on various situations. The results show that our architecture outperforms the other multimodal fusion architectures when some parts of data are not available.