mlsanityData to 7 Oct 2026

Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performance

arXiv 2410.16261 ↗First posted 21 Oct 2024cs.CVcited by 94

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Multimodal large language models (MLLMs) have demonstrated impressive performance in vision-language tasks across a broad spectrum of domains. However, the large model scale and associated high computational costs pose significant challenges for training and deploying MLLMs on consumer-grade GPUs or edge devices, thereby hindering their widespread application. In this work, we introduce Mini-InternVL, a series of MLLMs with parameters ranging from 1B to 4B, which achieves 90% of the performance with only 5% of the parameters. This significant improvement in efficiency and effectiveness makes our models more accessible and applicable in various real-world scenarios. To further promote the adoption of our models, we develop a unified adaptation framework for Mini-InternVL, which enables our models to transfer and outperform specialized models in downstream tasks, including autonomous driving, medical images, and remote sensing. We believe that our study can provide valuable insights and resources to advance the development of efficient and effective MLLMs. Code is available at https://github.com/OpenGVLab/InternVL.

Key facts. Cited by 94 papers in the map (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 120 of its 151 references that are in the map. First posted on arXiv 21 Oct 2024.

Builds on

120 of 151 references.

Closest ancestors · 16

Same direction, newest first

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution18 Sep
  2. 2024Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders28 Aug
  3. 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?23 Aug
  4. 2024LLaVA-OneVision: Easy Visual Task Transfer6 Aug
  5. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phone3 Aug
  6. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs24 Jun
  7. 2024Needle In A Multimodal Haystack11 Jun
  8. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites25 Apr
  9. 2024Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models5 Mar
  10. 2024Efficient Multimodal Learning from Data-centric Perspective18 Feb
  11. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs11 Jan
  12. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks21 Dec
4 more
  1. 2023VILA: On Pre-training for Visual Language Models12 Dec
  2. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captions21 Nov
  3. 2023Improved Baselines with Visual Instruction Tuning5 Oct
  4. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul

Other references · 104

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2023Segment Anythingcited by 8,158
  8. 2023Visual Instruction Tuningcited by 7,792
  9. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  10. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  11. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  12. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
38 more
  1. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  2. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  3. 2023Qwen Technical Reportcited by 3,273
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  5. 2023Mistral 7Bcited by 3,065
  6. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  7. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  8. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  9. 2024Qwen2 Technical Reportcited by 2,076
  10. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  11. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  12. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  13. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  14. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  15. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  16. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  17. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  18. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  19. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  20. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  21. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  22. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  23. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
  24. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  25. 2016A Diagram Is Worth A Dozen Imagescited by 793
  26. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  27. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  28. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  29. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
  30. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  31. 2024PaliGemma: A versatile 3B VLM for transfercited by 679
  32. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 659
  33. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
  34. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
  35. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
  36. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 443
  37. 2021InfographicVQAcited by 430
  38. 2021OCR-free Document Understanding Transformercited by 400

50 most cited of 104 All 104 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 59; 2026: 26.

94 papers.

Sort:

By area:

  1. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  2. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  3. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  4. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learningcited by 398
  5. 2024GME: Improving Universal Multimodal Retrieval by Multimodal LLMscited by 166
  6. 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignmentcited by 149
  7. 2025VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Modelscited by 102
  8. 2024DyCoke: Dynamic Compression of Tokens for Fast Video Large Language Modelscited by 93
  9. 2025HoliTom: Holistic Token Merging for Fast Video Large Language Modelscited by 58
  10. 2025TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Modelcited by 57
  11. 2025Scalable Vision Language Model Training via High Quality Data Curationcited by 54
  12. 2025FastVID: Dynamic Density Pruning for Fast Video Large Language Modelscited by 47
38 more
  1. 2025MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoningcited by 45
  2. 2025Evo-1: Lightweight Vision-Language-Action Model with Preserved Semantic Alignmentcited by 35
  3. 2025CAFe: Unifying Representation and Generation with Contrastive-Autoregressive Finetuningcited by 31
  4. 2024V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encodingcited by 28
  5. 2025A Survey of Token Compression for Efficient Multimodal Large Language Modelscited by 22
  6. 2025YoChameleon: Personalized Vision and Language Generationcited by 20
  7. 2025Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimizationcited by 20
  8. 2025Docopilot: Improving Multimodal Models for Document-Level Understandingcited by 19
  9. 2024MolParser: End-to-end Visual Recognition of Molecule Structures in the Wildcited by 17
  10. 2025Large Language Models for Computer-Aided Design: A Surveycited by 16
  11. 2025HiDe: Rethinking The Zoom-IN method in High Resolution MLLMs via Hierarchical Decouplingcited by 16
  12. 2025FisherTune: Fisher-Guided Robust Tuning of Vision Foundation Models for Domain Generalized Segmentationcited by 15
  13. 2024Do Language Models Understand Time?cited by 14
  14. 2025LanP: Rethinking the Impact of Language Priors in Large Vision-Language Modelscited by 14
  15. 2026HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Bettercited by 14
  16. 2025Mono-InternVL-1.5: Towards Cheaper and Faster Monolithic Multimodal Large Language Modelscited by 13
  17. 2025Consensus Entropy: Harnessing Multi-VLM Agreement for Self-Verifying and Self-Improving OCRcited by 12
  18. 2024MVTamperBench: Evaluating Robustness of Vision-Language Modelscited by 11
  19. 2025Pushing the Limits of Safety: A Technical Report on the ATLAS Challenge 2025cited by 11
  20. 2026Lance: Unified Multimodal Modeling by Multi-Task Synergycited by 11
  21. 2025MedReasoner: Reinforcement Learning Drives Reasoning Grounding from Clinical Thought to Pixel-Level Precisioncited by 10
  22. 2025DriveQA: Passing the Driving Knowledge Testcited by 10
  23. 2025Map the Flow: Revealing Hidden Pathways of Information in VideoLLMscited by 9
  24. 2026Unifying Language-Action Understanding and Generation for Autonomous Drivingcited by 9
  25. 2025LED: LLM Enhanced Open-Vocabulary Object Detection without Human Curated Data Generationcited by 8
  26. 2025Octopus: Agentic Multimodal Reasoning with Six-Capability Orchestrationcited by 8
  27. 2024SCBench: A Sports Commentary Benchmark for Video LLMscited by 7
  28. 2025Visual Instruction Pretraining for Domain-Specific Foundation Modelscited by 7
  29. 2025MaTVLM: Hybrid Mamba-Transformer for Efficient Vision-Language Modelingcited by 6
  30. 2025GraphPilot: Grounded Scene Graph Conditioning for Language-Based Autonomous Drivingcited by 6
  31. 2026LLMind: Bio-inspired Training-free Adaptive Visual Representations for Vision-Language Modelscited by 6
  32. 2026LMMs Meet Object-Centric Vision: Understanding, Segmentation, Editing and Generationcited by 6
  33. 2025Multimodal Fake News Detection: MFND Dataset and Shallow-Deep Multitask Learningcited by 5
  34. 2024MMGenBench: Fully Automatically Evaluating LMMs from the Text-to-Image Generation Perspectivecited by 4
  35. 2025DADM: Dual Alignment of Domain and Modality for Face Anti-spoofingcited by 4
  36. 2025Evaluation of Safety Cognition Capability in Vision-Language Models for Autonomous Drivingcited by 4
  37. 2025MMAFFBen: A Multilingual and Multimodal Affective Analysis Benchmark for Evaluating LLMs and VLMscited by 4
  38. 2025LiteVLM: A Low-Latency Vision-Language Model Inference Pipeline for Resource-Constrained Environmentscited by 4

50 most cited of 94 All 94 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Xmodel-VLM: A Simple Baseline for Multimodal Vision Language Model
  2. 2025Benchmarking Large and Small MLLMs
  3. 2025DriVLM: Domain Adaptation of Vision-Language Models in Autonomous Driving
  4. 2023MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devices
  5. 2025SmolVLM: Redefining small and efficient multimodal models
  6. 2023TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones
  7. 2024A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Tasks
  8. 2025Small Vision-Language Models: A Survey on Compact Architectures and Techniques