Data to 14 Oct 2026

CentralNet: a Multilayer Approach for Multimodal Fusion

arXiv 1808.07275 ↗First posted 22 Aug 2018cs.AI cs.CV cs.MMcited by 86

Not on the map. Why?

Abstract (arXiv)

This paper proposes a novel multimodal fusion approach, aiming to produce best possible decisions by integrating information coming from multiple media. While most of the past multimodal approaches either work by projecting the features of different modalities into the same space, or by coordinating the representations of each modality through the use of constraints, our approach borrows from both visions. More specifically, assuming each modality can be processed by a separated deep convolutional network, allowing to take decisions independently from each modality, we introduce a central network linking the modality specific networks. This central network not only provides a common feature embedding but also regularizes the modality specific networks through the use of multi-task learning. The proposed approach is validated on 4 different computer vision tasks on which it consistently improves the accuracy of existing multimodal fusion approaches.

Key facts. Cited by 86 papers in the map (data through 14 Oct 2026). First posted on arXiv 22 Aug 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 8; 2022: 17; 2023: 15; 2024: 23; 2025: 18; 2026: 5.

86 papers.

Sort:

By area:

2021SMIL: Multimodal Learning with Severely Missing Modalitycited by 245
2022Are Multimodal Transformers Robust to Missing Modality?cited by 186
2023Multi-modal Learning with Missing Modality via Shared-Specific Feature Modellingcited by 163
2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 146
2022PMR: Prototypical Modal Rebalance for Multimodal Learningcited by 116
2024Deep Multimodal Learning with Missing Modality: A Surveycited by 92
2023Boosting Multi-modal Model Performance with Adaptive Gradient Modulationcited by 73
2022Dynamic Multimodal Fusioncited by 55
2022The Modality Focusing Hypothesis: Towards Understanding Crossmodal Knowledge Distillationcited by 44
2023Enhancing multimodal cooperation via sample-level modality valuationcited by 44
2021N24News: A New Dataset for Multimodal News Classificationcited by 40
2021Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directionscited by 35

38 more

2024Classifier-guided Gradient Modulation for Enhanced Multimodal Learningcited by 35
2024On-the-fly Modulation for Balanced Multimodal Learningcited by 32
2022High-Modality Multimodal Transformer: Quantifying Modality & Interaction Heterogeneity for High-Modality Representation Learningcited by 31
2024Coupled Mamba: Enhanced Multi-modal Fusion with Coupled State Space Modelcited by 31
2025A Closer Look at Multimodal Representation Collapsecited by 28
2023Multi-modal Machine Learning in Engineering Design: A Review and Future Directionscited by 27
2025SimMLM: A Simple Framework for Multi-modal Learning with Missing Modalitycited by 23
2021Revisiting spatio-temporal layouts for compositional action recognitioncited by 19
2023VaQuitA: Enhancing Alignment in LLM-Assisted Video Understandingcited by 18
2024Improving Multimodal Learning with Multi-Loss Gradient Modulationcited by 18
2024Learning to Rebalance Multi-Modal Optimization by Adaptively Masking Subnetworkscited by 16
2024Balancing Multimodal Training Through Game-Theoretic Regularizationcited by 15
2021BM-NAS: Bilevel Multimodal Neural Architecture Searchcited by 13
2022SANCL: Multimodal Review Helpfulness Prediction with Selective Attention and Natural Contrastive Learningcited by 12
2022AWSnet: An Auto-weighted Supervision Attention Network for Myocardial Scar and Edema Segmentation in Multi-sequence Cardiac Magnetic Resonance Imagescited by 10
2021Multimodal Fusion Refiner Networkscited by 9
2023Multimodal deep representation learning for quantum cross-platform verificationcited by 9
2023Harmonic-NAS: Hardware-Aware Multimodal Neural Architecture Search on Resource-constrained Devicescited by 8
2024Jointly Modeling Inter- & Intra-Modality Dependencies for Multi-modal Learningcited by 8
2025Improving Multimodal Learning Balance and Sufficiency through Data Remixingcited by 8
2023Multi-modal Machine Learning for Vehicle Rating Predictions Using Image, Text, and Parametric Datacited by 7
2023Semantic-Aware Frame-Event Fusion based Pattern Recognition via Large Vision-Language Modelscited by 7
2024Bag of Tricks for Multimodal AutoML with Image, Text, and Tabular Datacited by 7
2025The More, the Merrier: Contrastive Fusion for Higher-Order Multimodal Alignmentcited by 7
2024CrisisKAN: Knowledge-infused and Explainable Multimodal Attention Network for Crisis Event Classificationcited by 6
2022Multi-Modal Attribute Extraction for E-Commercecited by 5
2022Common Practices and Taxonomy in Deep Multi-view Fusion for Remote Sensing Applicationscited by 5
2023What Makes for Robust Multi-Modal Models in the Face of Missing Modalities?cited by 5
2023MOSEL: Inference Serving Using Dynamic Modality Selectioncited by 5
2024Credibility-Aware Multi-Modal Fusion Using Probabilistic Circuitscited by 5
2024Guided Masked Self-Distillation Modeling for Distributed Multimedia Sensor Event Analysiscited by 5
2024Meta-Learned Modality-Weighted Knowledge Distillation for Robust Multi-Modal Learning with Missing Datacited by 5
2025G2D: Boosting Multimodal Learning with Gradient-Guided Distillationcited by 5
2025MST-Distill: Mixture of Specialized Teachers for Cross-Modal Knowledge Distillationcited by 5
2025Partial Information Decomposition via Normalizing Flows in Latent Gaussian Distributionscited by 5
2022Language Model-Based Paired Variational Autoencoders for Robotic Language Learningcited by 4
2022Brainish: Formalizing A Multimodal Language for Intelligence and Consciousnesscited by 4
2023MultiZoo & MultiBench: A Standardized Toolkit for Multimodal Deep Learningcited by 4

50 most cited of 86 All 86 (JSON)