mlsanityData to 7 Oct 2026

InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output

arXiv 2407.03320 ↗First posted 3 Jul 2024cs.CV cs.CLcited by 195

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

We present InternLM-XComposer-2.5 (IXC-2.5), a versatile large-vision language model that supports long-contextual input and output. IXC-2.5 excels in various text-image comprehension and composition applications, achieving GPT-4V level capabilities with merely 7B LLM backend. Trained with 24K interleaved image-text contexts, it can seamlessly extend to 96K long contexts via RoPE extrapolation. This long-context capability allows IXC-2.5 to excel in tasks requiring extensive input and output contexts. Compared to its previous 2.0 version, InternLM-XComposer-2.5 features three major upgrades in vision-language comprehension: (1) Ultra-High Resolution Understanding, (2) Fine-Grained Video Understanding, and (3) Multi-Turn Multi-Image Dialogue. In addition to comprehension, IXC-2.5 extends to two compelling applications using extra LoRA parameters for text-image composition: (1) Crafting Webpages and (2) Composing High-Quality Text-Image Articles. IXC-2.5 has been evaluated on 28 benchmarks, outperforming existing open-source state-of-the-art models on 16 benchmarks. It also surpasses or competes closely with GPT-4V and Gemini Pro on 16 key tasks. The InternLM-XComposer-2.5 is publicly available at https://github.com/InternLM/InternLM-XComposer.

Key facts. Cited by 195 papers in the map (data through 7 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 131 of its 161 references that are in the map. First posted on arXiv 3 Jul 2024.

Builds on

131 of 161 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Long Context Transfer from Language to Vision24 Jun
  2. 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding20 Jun
  3. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May
  4. 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning25 Apr
  5. 2024From Image to Video, what do we need in multimodal LLMs?18 Apr
  6. 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding8 Apr
  7. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models28 Nov
  8. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmark28 Nov
  9. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projection16 Nov
  10. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding5 Jun

Other references · 121

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2017Proximal Policy Optimization Algorithmscited by 14,441
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  9. 2023Visual Instruction Tuningcited by 7,792
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  11. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  12. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
38 more
  1. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  2. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  3. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  4. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  5. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  6. 2015VQA: Visual Question Answeringcited by 2,489
  7. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  8. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  9. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  10. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  11. 2019Towards VQA Models That Can Readcited by 1,696
  12. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  13. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  14. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  15. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  16. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  17. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  18. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  19. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  20. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
  21. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  22. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  23. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  24. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  25. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
  26. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  27. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 891
  28. 2023VideoChat: Chat-Centric Video Understandingcited by 851
  29. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  30. 2016A Diagram Is Worth A Dozen Imagescited by 793
  31. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  32. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
  33. 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
  34. 2023Baichuan 2: Open Large-scale Language Modelscited by 664
  35. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 560
  36. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 548
  37. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
  38. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509

50 most cited of 121 All 121 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 59; 2025: 103; 2026: 33.

195 papers.

Sort:

By area:

  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  2. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
  3. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  4. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
  5. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
  6. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 321
  7. 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Modelscited by 285
  8. 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reductioncited by 283
  9. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
  10. 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 212
  11. 2024Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Searchcited by 171
  12. 2024Building and better understanding vision-language models: insights and future directionscited by 169
38 more
  1. 2025WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMscited by 147
  2. 2024AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmarkcited by 137
  3. 2024Improve Vision Language Model Chain-of-thought Reasoningcited by 135
  4. 2024Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Inputcited by 132
  5. 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 122
  6. 2024Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Modelscited by 112
  7. 2024MMSearch: Benchmarking the Potential of Large Models as Multi-modal Search Enginescited by 98
  8. 2024TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Modelscited by 96
  9. 2024Video-CCAM: Enhancing Video-Language Understanding with Causal Cross-Attention Masks for Short and Long Videoscited by 95
  10. 2024Apollo: An Exploration of Video Understanding in Large Multimodal Modelscited by 87
  11. 2024MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuningcited by 75
  12. 2024Multimodal Alignment and Fusion: A Surveycited by 73
  13. 2025InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Modelcited by 73
  14. 2025Transformers without Normalizationcited by 69
  15. 2025VideoRoPE: What Makes for Good Video Rotary Position Embedding?cited by 63
  16. 2024ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Timecited by 57
  17. 2025GeoPixel: Pixel Grounding Large Multimodal Model in Remote Sensingcited by 55
  18. 2025SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understandingcited by 55
  19. 2025Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Modelscited by 54
  20. 2025Visual Agentic Reinforcement Fine-Tuningcited by 53
  21. 2025SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversioncited by 51
  22. 2024Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoningcited by 50
  23. 2025XLRS-Bench: Could Your Multimodal LLMs Understand Extremely Large Ultra-High-Resolution Remote Sensing Imagery?cited by 50
  24. 2024The Curse of Multi-Modalities: Evaluating Hallucinations of Large Multimodal Models across Language, Visual, and Audiocited by 48
  25. 2024InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactionscited by 48
  26. 2025TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMscited by 46
  27. 2024AIM: Adaptive Inference of Multi-Modal LLMs via Token Merging and Pruningcited by 43
  28. 2025Seeing Far and Clearly: Mitigating Hallucinations in MLLMs with Attention Causal Decodingcited by 42
  29. 2025VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learningcited by 41
  30. 2025ViSpeak: Visual Instruction Feedback in Streaming Videoscited by 38
  31. 2024Vinoground: Scrutinizing LMMs over Dense Temporal Reasoning with Short Videoscited by 37
  32. 2025A Survey of Multimodal Retrieval-Augmented Generationcited by 37
  33. 2024Lost in Time: A New Temporal Benchmark for VideoLLMscited by 36
  34. 2025What Kind of Visual Tokens Do We Need? Training-free Visual Token Pruning for Multi-modal Large Language Models from the Perspective of Graphcited by 35
  35. 2025When Large Vision-Language Model Meets Large Remote Sensing Imagery: Coarse-to-Fine Text-Guided Token Pruningcited by 35
  36. 2024MIA-DPO: Multi-Image Augmented Direct Preference Optimization For Large Vision-Language Modelscited by 34
  37. 2025Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual Reasoningcited by 34
  38. 2024Dynamic-VLM: Simple Dynamic Visual Token Compression for VideoLLMcited by 31

50 most cited of 195 All 195 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges
  2. 2025LongWriter-V: Enabling Ultra-Long and High-Fidelity Generation in Vision-Language Models
  3. 2024X-Prompt: Towards Universal In-Context Image Generation in Auto-Regressive Vision Language Foundation Models
  4. 2026Training Long-Context Vision-Language Models Effectively with Generalization Beyond 128K Context
  5. 2025MMLongBench: Benchmarking Long-Context Vision-Language Models Effectively and Thoroughly
  6. 2025MMLongCite: A Benchmark for Evaluating Fidelity of Long-Context Vision-Language Models
  7. 2023ReForm-Eval: Evaluating Large Vision Language Models via Unified Re-Formulation of Task-Oriented Benchmarks
  8. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects