# MLLM training and visual instruction tuning

Area: [Multimodal LLMs / vision-language models](https://mlsanity.com/mllm/index.md). HTML: https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/

## Key facts

- MLLM training and visual instruction tuning: 1,468 papers on arXiv since 2021 (data through 7 Oct 2026).
- 336 of its 1,468 mapped papers (23%) cite “Improved Baselines with Visual Instruction Tuning” (2023, arXiv 2310.03744).
- 272 of its 1,468 mapped papers (19%) cite “InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning” (2023, arXiv 2305.06500).
- 22 new papers 17 Sep–7 Oct.

## Most cited here

- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023), cited by 336 papers here
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023), cited by 272 papers here
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023), cited by 270 papers here
- [MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2306.13394/) (2023), cited by 201 papers here
- [MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities](https://mlsanity.com/paper/2308.02490/) (2023), cited by 169 papers here
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024), cited by 160 papers here
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023), cited by 148 papers here
- [SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension](https://mlsanity.com/paper/2307.16125/) (2023), cited by 130 papers here
- [ShareGPT4V: Improving Large Multi-Modal Models with Better Captions](https://mlsanity.com/paper/2311.12793/) (2023), cited by 116 papers here
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024), cited by 112 papers here

## New (17 Sep–7 Oct)

- [From Pixel to Coding: Evaluating the Figure Reproduction Capabilities of MLLMs](https://mlsanity.com/paper/2610.10066/) (2026-10-07)
- [Multimodal LLMs Can Learn to Read Brain Signals: A Vision--Language Model for Unified Multi-Task EEG Decoding](https://mlsanity.com/paper/2610.09355/) (2026-10-07)
- [Two Vectors Replace In-Context Demos: Structured Task Adaptation via Embeddings](https://mlsanity.com/paper/2610.07572/) (2026-10-06)
- [A Strong Baseline for Evaluating Vision Encoders in Multimodal Large Language Models](https://mlsanity.com/paper/2610.05413/) (2026-10-04)
- [Strong Helps Weak: Directional Cross-Modal Alignment Transfer in Multi-modal LLMs](https://mlsanity.com/paper/2610.04580/) (2026-10-03)
- [Architecture-Dependent Fusion Pathways in MLLMs](https://mlsanity.com/paper/2610.03289/) (2026-10-02)
- [Revealing Epistemic Uncertainty in MLLMs via Causal-Invariant Masking](https://mlsanity.com/paper/2610.02887/) (2026-10-02)
- [Slow-Fast Multi-Teacher On-Policy Distillation for Capability Preservation](https://mlsanity.com/paper/2610.02324/) (2026-10-01)
- [Gestalt: Large Multimodal Interplay Model](https://mlsanity.com/paper/2610.00576/) (2026-09-30)
- [MCD: Causal Distillation of Multimodal In-Context Learning in Large Vision-Language Models](https://mlsanity.com/paper/2609.39920/) (2026-09-30)
- [MM-FinEval: A Multi-Task Multimodal Benchmark for Real-World Financial Forecasting](https://mlsanity.com/paper/2609.38523/) (2026-09-29)
- [ReCAP: Retrieval-Guided Capability Reuse for Multimodal Continual Instruction Tuning](https://mlsanity.com/paper/2609.37889/) (2026-09-29)
- [Calibrate the Decisions That Change the Future: On-Policy Post-Training Quantization for Multimodal Large Language Models](https://mlsanity.com/paper/2609.36828/) (2026-09-29)
- [How Far Are We from Removing the Visual Encoder? Scaling Laws for Encoder-Free Multimodal Pretraining](https://mlsanity.com/paper/2609.35457/) (2026-09-28)
- [MoR-MLLM: Mixture of Recursions for Efficient Multimodal Large Language Models](https://mlsanity.com/paper/2610.08830/) (2026-09-27)
- [FORUM: Frozen Outputs Reconciled Using Model Agreement for Visual Grounding](https://mlsanity.com/paper/2609.37488/) (2026-09-27)
- [BIRD: Distilling Decision Boundaries into Rationales for MLLM Adaptation](https://mlsanity.com/paper/2609.33713/) (2026-09-27)
- [The Alignment Illusion in Multimodal Large Language Models](https://mlsanity.com/paper/2609.30210/) (2026-09-24)
- [Virtual Encoders in Multimodal Transformers](https://mlsanity.com/paper/2609.26513/) (2026-09-22)
- [BAS-OPD: Budget-Aware Selective On-Policy Self-Distillation for Fine-Grained Multimodal Perception](https://mlsanity.com/paper/2609.25891/) (2026-09-22)
- [Pretraining of Medical Visual Encoders Toward Multi-modal Large Language Models](https://mlsanity.com/paper/2609.23860/) (2026-09-20)
- [Offline Multimodal Large Language Models for Decision Support in Air Operations](https://mlsanity.com/paper/2609.21390/) (2026-09-18)