mlsanityData to 7 Oct 2026

Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution

arXiv 2409.12191 ↗First posted 18 Sep 2024cs.CV cs.AI cs.CLcited by 4,151

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

We present the Qwen2-VL Series, an advanced upgrade of the previous Qwen-VL models that redefines the conventional predetermined-resolution approach in visual processing. Qwen2-VL introduces the Naive Dynamic Resolution mechanism, which enables the model to dynamically process images of varying resolutions into different numbers of visual tokens. This approach allows the model to generate more efficient and accurate visual representations, closely aligning with human perceptual processes. The model also integrates Multimodal Rotary Position Embedding (M-RoPE), facilitating the effective fusion of positional information across text, images, and videos. We employ a unified paradigm for processing both images and videos, enhancing the model's visual perception capabilities. To explore the potential of large multimodal models, Qwen2-VL investigates the scaling laws for large vision-language models (LVLMs). By scaling both the model size-with versions at 2B, 8B, and 72B parameters-and the amount of training data, the Qwen2-VL Series achieves highly competitive performance. Notably, the Qwen2-VL-72B model achieves results comparable to leading models such as GPT-4o and Claude3.5-Sonnet across various multimodal benchmarks, outperforming other generalist models. Code is available at https://github.com/QwenLM/Qwen2-VL .

Key facts. Cited by 4,151 papers in the map (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 77 of its 120 references that are in the map. First posted on arXiv 18 Sep 2024.

Builds on

77 of 120 references.

Closest ancestors · 15

Same direction, newest first

  1. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phone3 Aug
  2. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites25 Apr
  3. 2024MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI24 Apr
  4. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captions21 Nov
  5. 2023Vision-Language Instruction Tuning: A Review and Analysis14 Nov
  6. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Models13 Nov
  7. 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Models11 Nov
  8. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration7 Nov
  9. 2023Improved Baselines with Visual Instruction Tuning5 Oct
  10. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)29 Sep
  11. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities4 Aug
  12. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul
3 more
  1. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic27 Jun
  2. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models23 Jun
  3. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning11 May

Other references · 62

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2024The Llama 3 Herd of Modelscited by 14,192
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  8. 2023Visual Instruction Tuningcited by 7,792
  9. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  10. 2016Layer Normalizationcited by 5,579
  11. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  12. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
38 more
  1. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  2. 2023Qwen Technical Reportcited by 3,273
  3. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  4. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  5. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  6. 2024Qwen2 Technical Reportcited by 2,076
  7. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  8. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  9. 2017Decoupled Weight Decay Regularizationcited by 1,722
  10. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  11. 2019Root Mean Square Layer Normalizationcited by 1,425
  12. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  13. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  14. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  15. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  16. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  17. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  18. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
  19. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  20. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  21. 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
  22. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
  23. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 572
  24. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  25. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 516
  26. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 507
  27. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
  28. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 466
  29. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
  30. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  31. 2023Perception Test: A Diagnostic Benchmark for Multimodal Video Modelscited by 310
  32. 2022Reducing Activation Recomputation in Large Transformer Modelscited by 300
  33. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  34. 2023Data Filtering Networkscited by 247
  35. 2024GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devicescited by 190
  36. 2023You Only Look at Screens: Multimodal Chain-of-Action Agentscited by 185
  37. 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learningcited by 158
  38. 2024Android in the Zoo: Chain-of-Action-Thought for GUI Agentscited by 156

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 331; 2025: 2312; 2026: 1508.

4,151 papers.

Sort:

By area:

  1. 2025Qwen3-VL Technical Reportcited by 2,534
  2. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  3. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  4. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  5. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  6. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  7. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  8. 2025Qwen2.5-Omni Technical Reportcited by 704
  9. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 584
  10. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
  11. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 559
  12. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 515
38 more
  1. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 507
  2. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  3. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
  4. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
  5. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 446
  6. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 439
  7. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learningcited by 398
  8. 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 388
  9. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 373
  10. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agentscited by 370
  11. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 364
  12. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 321
  13. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learningcited by 315
  14. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 315
  15. 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 296
  16. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 286
  17. 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reductioncited by 283
  18. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 271
  19. 2025Improving Video Generation with Human Feedbackcited by 267
  20. 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuningcited by 248
  21. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 246
  22. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RLcited by 244
  23. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
  24. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 241
  25. 2024Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimizationcited by 238
  26. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documentscited by 237
  27. 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcementcited by 234
  28. 2025LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Trainingcited by 233
  29. 2025Motus: A Unified Latent Action World Modelcited by 231
  30. 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 226
  31. 2025SmolVLM: Redefining small and efficient multimodal modelscited by 226
  32. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 213
  33. 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 212
  34. 2025MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understandingcited by 212
  35. 2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 211
  36. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 209
  37. 2024NVILA: Efficient Frontier Visual Language Modelscited by 206
  38. 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligencecited by 206

50 most cited of 4,151 All 4,151 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models
  2. 2025QLIP: A Dynamic Quadtree Vision Prior Enhances MLLM Performance Without Retraining
  3. 2024VisualRWKV-HD and UHD: Advancing High-Resolution Processing for Visual Language Models
  4. 2024FastVLM: Efficient Vision Encoding for Vision Language Models
  5. 2026Not All Error Yields to Scale: Where Scaling Stops in Vision-Language Inference
  6. 2026Towards Efficient Large Vision-Language Models: A Comprehensive Survey on Inference Strategies
  7. 2026VIVAS: Vitalizing Visual Perception in VLM Pre-training via Vision-language Unified Autoregressive Supervision
  8. 2026ViQ: Text-Aligned Visual Quantized Representations at Any Resolution