mlsanityData to 14 Oct 2026

Multimodal Representation Learning and Fusion

arXiv 2506.20494 ↗First posted 25 Jun 2025cs.LG cs.MMcited by 3

Not on the map. Why?

Abstract (arXiv)

Multi-modal learning is a fast growing area in artificial intelligence. It tries to help machines understand complex things by combining information from different sources, like images, text, and audio. By using the strengths of each modality, multi-modal learning allows AI systems to build stronger and richer internal representations. These help machines better interpretation, reasoning, and making decisions in real-life situations. This field includes core techniques such as representation learning (to get shared features from different data types), alignment methods (to match information across modalities), and fusion strategies (to combine them by deep learning models). Although there has been good progress, some major problems still remain. Like dealing with different data formats, missing or incomplete inputs, and defending against adversarial attacks. Researchers now are exploring new methods, such as unsupervised or semi-supervised learning, AutoML tools, to make models more efficient and easier to scale. And also more attention on designing better evaluation metrics or building shared benchmarks, make it easier to compare model performance across tasks and domains. As the field continues to grow, multi-modal learning is expected to improve many areas: computer vision, natural language processing, speech recognition, and healthcare. In the future, it may help to build AI systems that can understand the world in a way more like humans, flexible, context aware, and able to deal with real-world complexity.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 65 of its 74 references that are in the map. First posted on arXiv 25 Jun 2025.

Builds on

65 of 74 references.

References · 65

  1. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,013
  2. 2023Multimodal Deep Learningcited by 258
  3. 2021SMIL: Multimodal Learning with Severely Missing Modalitycited by 245
  4. 2023Multimodal Large Language Models: A Surveycited by 195
  5. 2018Learning Factorized Multimodal Representationscited by 171
  6. 2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 146
  7. 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
  8. 2024Deep Multimodal Learning with Missing Modality: A Surveycited by 92
  9. 2019MFAS: Multimodal Fusion Architecture Searchcited by 84
  10. 2024A Survey on Benchmarks of Multimodal Large Language Modelscited by 82
  11. 2024Multimodal Alignment and Fusion: A Surveycited by 74
  12. 2023Boosting Multi-modal Model Performance with Adaptive Gradient Modulationcited by 73
38 more
  1. 2024MMPareto: Boosting Multimodal Learning with Innocent Unimodal Assistancecited by 73
  2. 2018Automated Machine Learning: From Principles to Practicescited by 62
  3. 2024A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Taskscited by 57
  4. 2020Survey on Deep Multi-modal Data Analytics: Collaboration, Rivalry and Fusioncited by 53
  5. 2023On Uni-Modal Feature Learning in Supervised Multi-Modal Learningcited by 53
  6. 2022Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representationscited by 48
  7. 2022A survey of multimodal deep generative modelscited by 47
  8. 2021Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directionscited by 35
  9. 2022Modular and Parameter-Efficient Multimodal Fusion with Promptingcited by 35
  10. 2020Deep Multimodal Neural Architecture Searchcited by 34
  11. 2018Learning with privileged information via adversarial discriminative modality distillationcited by 30
  12. 2019EmbraceNet: A robust deep learning architecture for multimodal classificationcited by 30
  13. 2024AutoGluon-Multimodal (AutoMM): Supercharging Multimodal AutoML with Foundation Modelscited by 30
  14. 2022A Review on Methods and Applications in Multimodal Deep Learningcited by 29
  15. 2023Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptationcited by 29
  16. 2023Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applicationscited by 27
  17. 2024A Survey on Multimodal Benchmarks: In the Era of Large AI Modelscited by 27
  18. 2021Recent Advances and Trends in Multimodal Deep Learning: A Reviewcited by 26
  19. 2021Benchmarking Multimodal AutoML for Tabular Data with Text Fieldscited by 26
  20. 2023A Theory of Multimodal Learningcited by 25
  21. 2024A Systematic Review of Intermediate Fusion in Multimodal Deep Learning for Biomedical Applicationscited by 25
  22. 2021Multimodal Semi-Supervised Learning for 3D Objectscited by 24
  23. 2019Continual and Multi-Task Architecture Searchcited by 22
  24. 2021Revisit Multimodal Meta-Learning through the Lens of Multi-Task Learningcited by 22
  25. 2023Multimodal Learning Without Labeled Multimodal Data: Guarantees and Applicationscited by 21
  26. 2023Multimodality of AI for Education: Towards Artificial General Intelligencecited by 21
  27. 2024The Evolution of Multimodal Model Architecturescited by 21
  28. 2024HEMM: Holistic Evaluation of Multimodal Foundation Modelscited by 21
  29. 2023MM-BigBench: Evaluating Multimodal Models on Multimodal Content Comprehension Taskscited by 18
  30. 2024Survey of Large Multimodal Model Datasets, Application Categories and Taxonomycited by 17
  31. 2024Balancing Multimodal Training Through Game-Theoretic Regularizationcited by 15
  32. 2018Multimodal Grounding for Language Processingcited by 14
  33. 2024AutoM3L: An Automated Multimodal Machine Learning Framework with Large Language Modelscited by 13
  34. 2019Automated Machine Learning in Practice: State of the Art and Recent Resultscited by 12
  35. 2023Learning Unseen Modality Interactioncited by 12
  36. 2020Automated Machine Learning -- a brief review at the end of the early yearscited by 9
  37. 2023ChEF: A Comprehensive Evaluation Framework for Standardized Assessment of Multimodal Large Language Modelscited by 9
  38. 2024Attribution Regularization for Multimodal Paradigmscited by 7

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025New Spiking Architecture for Multi-Modal Decision-Making in Autonomous Vehiclescited by 2
  2. 2026CNN-ViT Fusion with Adaptive Attention Gate for Brain Tumor MRI Classification: A Hybrid Deep Learning Modelcited by 0
  3. 2026DAIN: Dynamic Agent-Based Interaction Network for Efficient and Collaborative Multimodal Reasoningcited by 0