mlsanityData to 7 Oct 2026

Grandes Modelos de Linguagem Multimodais (MLLMs): Da Teoria `a Pr'atica

arXiv 2602.12302 ↗First posted 11 Feb 2026cs.CL cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) combine the natural language understanding and generation capabilities of LLMs with perception skills in modalities such as image and audio, representing a key advancement in contemporary AI. This chapter presents the main fundamentals of MLLMs and emblematic models. Practical techniques for preprocessing, prompt engineering, and building multimodal pipelines with LangChain and LangGraph are also explored. For further practical study, supplementary material is publicly available online: https://github.com/neemiasbsilva/MLLMs-Teoria-e-Pratica. Finally, the chapter discusses the challenges and highlights promising trends.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 47 of its 65 references that are in the map. First posted on arXiv 11 Feb 2026.

Builds on

47 of 65 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024NVLM: Open Frontier-Class Multimodal LLMs17 Sep
  2. 2024Unveiling the Ignorance of MLLMs: Seeing Clearly, Answering Incorrectly15 Jun
  3. 2024The Revolution of Multimodal Large Language Models: A Survey19 Feb
  4. 2023Multimodal Large Language Models: A Survey22 Nov
  5. 2023A Survey on Multimodal Large Language Models23 Jun

Other references · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2023GPT-4 Technical Reportcited by 18,132
  7. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  8. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  9. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  10. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  11. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  12. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
30 more
  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023Visual Instruction Tuningcited by 7,792
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  6. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  7. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  8. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  9. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  10. 2022Emergent Abilities of Large Language Modelscited by 2,483
  11. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  12. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  13. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  14. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  15. 2014Show and Tell: A Neural Image Caption Generatorcited by 877
  16. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 326
  17. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 321
  18. 2018A Comprehensive Survey of Deep Learning for Image Captioningcited by 213
  19. 2025Why Language Models Hallucinatecited by 205
  20. 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognitioncited by 151
  21. 2023Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directionscited by 123
  22. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 74
  23. 2025How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inferencecited by 73
  24. 2024VideoEspresso: A Large-Scale Chain-of-Thought Dataset for Fine-Grained Video Reasoning via Core Frame Selectioncited by 64
  25. 2024Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Directioncited by 46
  26. 2025ECLeKTic: a Novel Challenge Set for Evaluation of Cross-Lingual Knowledge Transfercited by 28
  27. 2025Beyond English: The Impact of Prompt Translation Strategies across Languages and Tasks in Multilingual LLMscited by 22
  28. 2024VLRM: Vision-Language Models act as Reward Models for Image Captioningcited by 18
  29. 2025MetaMind: Modeling Human Social Thoughts with Metacognitive Multi-Agent Systemscited by 12
  30. 2025Multimodal LLMs See Sentimentcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Multilingual Large Language Models: A Systematic Survey
  2. 2024A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Tasks
  3. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspective
  4. 2024Survey of different Large Language Model Architectures: Trends, Benchmarks, and Challenges
  5. 2024A Survey on Evaluation of Multimodal Large Language Models
  6. 2024A Survey on Benchmarks of Multimodal Large Language Models
  7. 2024MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMs
  8. 2024A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Tasks