mlsanityData to 14 Oct 2026

MFAS: Multimodal Fusion Architecture Search

arXiv 1903.06496 ↗First posted 15 Mar 2019cs.LG cs.CV cs.NEcited by 84

Not on the map. Why?

Abstract (arXiv)

We tackle the problem of finding good architectures for multimodal classification problems. We propose a novel and generic search space that spans a large number of possible fusion architectures. In order to find an optimal architecture for a given dataset in the proposed search space, we leverage an efficient sequential model-based exploration approach that is tailored for the problem. We demonstrate the value of posing multimodal fusion as a neural architecture search problem by extensive experimentation on a toy dataset and two other real multimodal datasets. We discover fusion architectures that exhibit state-of-the-art performance for problems with different domain and dataset size, including the NTU RGB+D dataset, the largest multi-modal action recognition dataset available.

Key facts. Cited by 84 papers in the map (data through 14 Oct 2026). First posted on arXiv 15 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 22; 2022: 22; 2023: 11; 2024: 16; 2025: 7; 2026: 6.

84 papers.

Sort:

By area:

  1. 2022Are Multimodal Transformers Robust to Missing Modality?cited by 186
  2. 2022Trusted Multi-View Classification with Dynamic Evidential Fusioncited by 168
  3. 2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 146
  4. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  5. 2022Characterizing and overcoming the greedy nature of learning in multi-modal deep neural networkscited by 95
  6. 2021Reconsidering Representation Alignment for Multi-view Clusteringcited by 57
  7. 2021Deep RGB-D Saliency Detection with Depth-Sensitive Attention and Automatic Multi-Modal Fusioncited by 56
  8. 2022Dynamic Multimodal Fusioncited by 55
  9. 2021Neural Feature Search for RGB-Infrared Person Re-Identificationcited by 50
  10. 2022Divert More Attention to Vision-Language Trackingcited by 47
  11. 2024What to align in multimodal contrastive learning?cited by 47
  12. 2021A Review on Explainability in Multimodal Deep Neural Netscited by 46
38 more
  1. 2021Improving Multi-Modal Learning with Uni-Modal Teacherscited by 41
  2. 2021AdaMML: Adaptive Multi-Modal Learning for Efficient Video Recognitioncited by 36
  3. 2022MultiViz: Towards Visualizing and Understanding Multimodal Modelscited by 36
  4. 2021VPN++: Rethinking Video-Pose embeddings for understanding Activities of Daily Livingcited by 35
  5. 2021Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directionscited by 35
  6. 2022High-Modality Multimodal Transformer: Quantifying Modality & Interaction Heterogeneity for High-Modality Representation Learningcited by 31
  7. 2025Understanding the Emergence of Multimodal Representation Alignmentcited by 31
  8. 2021MUFASA: Multimodal Fusion Architecture Search for Electronic Health Recordscited by 30
  9. 2021Towards a Collective Agenda on AI for Earth Science Data Analysiscited by 27
  10. 2023Multi-modal Machine Learning in Engineering Design: A Review and Future Directionscited by 27
  11. 2024A review of deep learning-based information fusion techniques for multimodal medical image classificationcited by 25
  12. 2021"Subverting the Jewtocracy": Online Antisemitism Detection Using Multimodal Deep Learningcited by 21
  13. 2021Revisiting spatio-temporal layouts for compositional action recognitioncited by 19
  14. 2022Learning Multimodal Data Augmentation in Feature Spacecited by 18
  15. 2023Calibrating Multimodal Learningcited by 17
  16. 2024Predictive Dynamic Fusioncited by 17
  17. 2021Fusion-GCN: Multimodal Action Recognition using Graph Convolutional Networkscited by 15
  18. 2022Deep Learning on Multimodal Sensor Data at the Wireless Edge for Vehicular Networkcited by 15
  19. 2025To Align or Not to Align: Strategic Multimodal Representation Alignment for Optimal Performancecited by 14
  20. 2021BM-NAS: Bilevel Multimodal Neural Architecture Searchcited by 13
  21. 2024Multi-Modality Co-Learning for Efficient Skeleton-based Action Recognitioncited by 12
  22. 2024Revisiting Multimodal Fusion for 3D Anomaly Detection from an Architectural Perspectivecited by 12
  23. 2021Regional Attention with Architecture-Rebuilt 3D Network for RGB-D Gesture Recognitioncited by 10
  24. 2024A Survey on Multimodal Wearable Sensor-based Human Action Recognitioncited by 10
  25. 2021Multimodal Fusion Refiner Networkscited by 9
  26. 2022Slimmable Domain Adaptationcited by 9
  27. 2023Multimodal deep representation learning for quantum cross-platform verificationcited by 9
  28. 2025Taming Vision-Language Models for Medical Image Analysis: A Comprehensive Reviewcited by 9
  29. 2023Harmonic-NAS: Hardware-Aware Multimodal Neural Architecture Search on Resource-constrained Devicescited by 8
  30. 2024Jointly Modeling Inter- & Intra-Modality Dependencies for Multi-modal Learningcited by 8
  31. 2024From CNNs to Transformers in Multimodal Human Action Recognition: A Surveycited by 7
  32. 2022Skeleton Sequence and RGB Frame Based Multi-Modality Feature Fusion Network for Action Recognitioncited by 6
  33. 2022Multi-Modal Attribute Extraction for E-Commercecited by 5
  34. 2022Common Practices and Taxonomy in Deep Multi-view Fusion for Remote Sensing Applicationscited by 5
  35. 2023MOSEL: Inference Serving Using Dynamic Modality Selectioncited by 5
  36. 2024Credibility-Aware Multi-Modal Fusion Using Probabilistic Circuitscited by 5
  37. 2024Guided Masked Self-Distillation Modeling for Distributed Multimedia Sensor Event Analysiscited by 5
  38. 2021Software/Hardware Co-design for Multi-modal Multi-task Learning in Autonomous Systemscited by 4

50 most cited of 84 All 84 (JSON)