mlsanityData to 14 Oct 2026

Surveying the MLLM Landscape: A Meta-Review of Current Surveys

arXiv 2409.18991 ↗First posted 17 Sep 2024cs.CLcited by 20

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

The rise of Multimodal Large Language Models (MLLMs) has become a transformative force in the field of artificial intelligence, enabling machines to process and generate content across multiple modalities, such as text, images, audio, and video. These models represent a significant advancement over traditional unimodal systems, opening new frontiers in diverse applications ranging from autonomous agents to medical diagnostics. By integrating multiple modalities, MLLMs achieve a more holistic understanding of information, closely mimicking human perception. As the capabilities of MLLMs expand, the need for comprehensive and accurate performance evaluation has become increasingly critical. This survey aims to provide a systematic review of benchmark tests and evaluation methods for MLLMs, covering key topics such as foundational concepts, applications, evaluation methodologies, ethical concerns, security, efficiency, and domain-specific applications. Through the classification and analysis of existing literature, we summarize the main contributions and methodologies of various surveys, conduct a detailed comparative analysis, and examine their impact within the academic community. Additionally, we identify emerging trends and underexplored areas in MLLM research, proposing potential directions for future studies. This survey is intended to offer researchers and practitioners a comprehensive understanding of the current state of MLLM evaluation, thereby facilitating further progress in this rapidly evolving field.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 75 of its 99 references that are in the map. First posted on arXiv 17 Sep 2024.

Builds on

75 of 99 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024A Survey on Benchmarks of Multimodal Large Language Models16 Aug
  2. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspective26 May
  3. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark7 Feb
  4. 2023Multimodal Large Language Models: A Survey22 Nov
  5. 2023A Survey on Multimodal Large Language Models23 Jun

Other references · 70

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  7. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  9. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  10. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  11. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  12. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
38 more
  1. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  2. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  3. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  5. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  6. 2018Deep contextualized word representationscited by 2,290
  7. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  8. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  9. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  10. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  11. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  12. 2019Contrastive Multiview Codingcited by 1,301
  13. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  14. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  15. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 808
  16. 2021Florence: A New Foundation Model for Computer Visioncited by 804
  17. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 647
  18. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  19. 2024Large Language Models: A Surveycited by 525
  20. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 411
  21. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 401
  22. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 368
  23. 2024A Survey on Hallucination in Large Vision-Language Modelscited by 357
  24. 2021An Empirical Study of Training End-to-End Vision-and-Language Transformerscited by 271
  25. 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 257
  26. 2023Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylizationcited by 254
  27. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 234
  28. 2019UNITER: UNiversal Image-TExt Representation Learningcited by 228
  29. 2023Video Understanding with Large Language Models: A Surveycited by 206
  30. 2014Unifying Visual-Semantic Embeddings with Multimodal Neural Language Modelscited by 198
  31. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 194
  32. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 192
  33. 2023Large Language Models for Generative Information Extraction: A Surveycited by 185
  34. 2023Large Language Models for Robotics: A Surveycited by 133
  35. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 110
  36. 2024Large Multimodal Agents: A Surveycited by 105
  37. 2023A Survey on Fairness in Large Language Modelscited by 95
  38. 2024Large Language Models and Games: A Survey and Roadmapcited by 93

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 5; 2026: 9.

20 papers.

Sort:

By area:

  1. 2025A Survey of Context Engineering for Large Language Modelscited by 126
  2. 2025Earth-Agent: Unlocking the Full Landscape of Earth Observation with Agentscited by 46
  3. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Modelscited by 34
  4. 2024Heuristic-Induced Multimodal Risk Distribution Jailbreak Attack for Multimodal Large Language Modelscited by 33
  5. 2024Large Language Model Benchmarks in Medical Taskscited by 17
  6. 2024From Word Vectors to Multimodal Embeddings: Techniques, Applications, and Future Directions For Large Language Modelscited by 6
  7. 2024Seeing is Deceiving: Exploitation of Visual Pathways in Multi-Modal Language Modelscited by 4
  8. 2024From Noise to Nuance: Advances in Deep Generative Image Modelscited by 3
  9. 2025Multimodal Large Language Models for End-to-End Affective Computing: Benchmarking and Boosting with Generative Knowledge Promptingcited by 3
  10. 2025Can Large Multimodal Models Actively Recognize Faulty Inputs? A Systematic Evaluation Framework of Their Input Scrutiny Abilitycited by 3
  11. 2026360deg Image Perception with MLLMs: A Comprehensive Benchmark and a Training-Free Methodcited by 3
  12. 2026LookWise: Knowing When and Where to Look for Fine-Grained Visual Reasoning in Multimodal Large Language Modelscited by 2
8 more
  1. 2026TouchThinker: Scaling Tactile Commonsense Reasoning to the Open World with Large-scale Data and Action-aware Representationcited by 2
  2. 2026TacReasoner: A Dynamic Tactile-Language Framework for Interactive Reasoning in Real-World Scenarioscited by 2
  3. 2025Enhancing Sports Strategy with Video Analytics and Data Mining: Assessing the effectiveness of Multimodal LLMs in tennis video analysiscited by 1
  4. 2026Spotlight and Shadow: Attention-Guided Dual-Anchor Introspective Decoding for MLLM Hallucination Mitigationcited by 1
  5. 2026RISE: Reliable Improvement in Self-Evolving Vision-Language Modelscited by 1
  6. 2026Adaptive Multi-Stage Patent Claim Generation with Unified Quality Assessmentcited by 0
  7. 2026DriveCode: Domain Specific Numerical Encoding for LLM-Based Autonomous Drivingcited by 0
  8. 2026Design-MLLM: A Reinforcement Alignment Framework for Verifiable and Aesthetic Interior Designcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Survey on Multimodal Benchmarks: In the Era of Large AI Models
  2. 2024A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Tasks
  3. 2024MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMs
  4. 2024A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Tasks
  5. 2023MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria
  6. 2025Multimodal Large Language Models for Medicine: A Comprehensive Survey
  7. 2024A Survey on Evaluation of Multimodal Large Language Models
  8. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning