mlsanityData to 14 Oct 2026

InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition

arXiv 2309.15112 ↗First posted 26 Sep 2023cs.CVcited by 297

Not on the map. Why?

Cited by 189 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose InternLM-XComposer, a vision-language large model that enables advanced image-text comprehension and composition. The innovative nature of our model is highlighted by three appealing properties: 1) Interleaved Text-Image Composition: InternLM-XComposer can effortlessly generate coherent and contextual articles that seamlessly integrate images, providing a more engaging and immersive reading experience. Simply provide a writing instruction, and our system will generate the corresponding manuscript. It can intelligently identify the areas in the text where images would enhance the content and automatically insert the most appropriate visual candidates. 2) Comprehension with Rich Multilingual Knowledge: The text-image comprehension is empowered by training on an extensive multi-modal multilingual database with carefully crafted strategies, resulting in a deep understanding of visual content. 3) State-of-the-art Performance: Our model consistently achieves state-of-the-art results across various mainstream benchmarks for vision-language foundational models, including MME Benchmark, MMBench, MMBench-CN, Seed-Bench, CCBench (Chinese Cultural Benchmark), QBench and Tiny LVLM. Owing to the absence of established metrics for quantitatively assessing text-image composition, we have devised a robust evaluation procedure that comprises both human and GPT4-Vision (GPT4-V) to ensure reliability. Notably, our InternLM-XComposer achieves competitive text-image composition scores compared to public solutions, including GPT4-V and GPT3.5. Collectively, InternLM-XComposer seamlessly blends advanced text-image comprehension and composition, revolutionizing vision-language interaction and offering new insights and opportunities. The InternLM-XComposer model series are publicly available at https://github.com/InternLM/InternLM-XComposer.

Key facts. Cited by 297 papers in the map (data through 14 Oct 2026). Builds on 64 of its 82 references that are in the map. First posted on arXiv 26 Sep 2023.

Builds on

64 of 82 references.

References · 64

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2023GPT-4 Technical Reportcited by 18,396
  6. 2022Training language models to follow instructions with human feedbackcited by 15,971
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  8. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  9. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  10. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  11. 2023Visual Instruction Tuningcited by 7,869
  12. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
38 more
  1. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  2. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  3. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  4. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  5. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  6. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  7. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  8. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  9. 2015VQA: Visual Question Answeringcited by 2,506
  10. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  11. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  12. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  13. 2019Towards VQA Models That Can Readcited by 1,718
  14. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  15. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  16. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  17. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  18. 2021Grounded Language-Image Pre-trainingcited by 1,156
  19. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,121
  20. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  21. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  22. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
  23. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 940
  24. 2023A Survey on Multimodal Large Language Modelscited by 909
  25. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  26. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
  27. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 691
  28. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
  29. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  30. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 565
  31. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 550
  32. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 537
  33. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuningcited by 445
  34. 2020TextCaps: a Dataset for Image Captioning with Reading Comprehensioncited by 400
  35. 2016Visual Dialogcited by 377
  36. 2022Visual Spatial Reasoningcited by 293
  37. 2022GLIPv2: Unifying Localization and Vision-Language Understandingcited by 275
  38. 2021IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoningcited by 267

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 25; 2024: 154; 2025: 93; 2026: 25.

297 papers.

Sort:

By area:

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  2. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  3. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  4. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  5. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
  6. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 567
  7. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 526
  8. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 521
  9. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
  10. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocationcited by 478
  11. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  12. 2024Mini-Gemini: Mining the Potential of Multi-modality Vision Language Modelscited by 365
38 more
  1. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 353
  2. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 331
  3. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  4. 2024MoE-LLaVA: Mixture of Experts for Large Vision-Language Modelscited by 300
  5. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 295
  6. 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 292
  7. 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Modelscited by 291
  8. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 278
  9. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimizationcited by 225
  10. 2025VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interactioncited by 197
  11. 2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Outputcited by 195
  12. 2024MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGIcited by 193
  13. 2024VITA: Towards Open-Source Interactive Omni Multimodal LLMcited by 173
  14. 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Modelscited by 167
  15. 2024Streaming Long Video Understanding with Large Language Modelscited by 167
  16. 2023ChartLlama: A Multimodal LLM for Chart Understanding and Generationcited by 163
  17. 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Modelscited by 161
  18. 2024InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HDcited by 159
  19. 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 159
  20. 2023Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Modelscited by 152
  21. 2023Alpha-CLIP: A CLIP Model Focusing on Wherever You Wantcited by 151
  22. 2024SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehensioncited by 146
  23. 2024DynaMath: A Dynamic Visual Benchmark for Evaluating Mathematical Reasoning Robustness of Vision Language Modelscited by 146
  24. 2025OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?cited by 142
  25. 2025Teaching Large Language Models to Regress Accurate Image Quality Scores using Score Distributioncited by 141
  26. 2024LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Modelcited by 138
  27. 2024Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Inputcited by 134
  28. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 132
  29. 2024Visual In-Context Learning for Large Vision-Language Modelscited by 128
  30. 2024OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understandingcited by 127
  31. 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 127
  32. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformationcited by 124
  33. 2023ChartBench: A Benchmark for Complex Visual Reasoning in Chartscited by 120
  34. 2024HourVideo: 1-Hour Video-Language Understandingcited by 113
  35. 2023How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMscited by 108
  36. 2023Mixture of Cluster-conditional LoRA Experts for Vision-language Instruction Tuningcited by 106
  37. 2023Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Modelscited by 105
  38. 2024A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Modelscited by 100

50 most cited of 297 All 297 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025ReadBench: Measuring the Dense Text Visual Reading Ability of Vision-Language Models
  2. 2024Cross-Lingual Text-Rich Visual Comprehension: An Information Theory Perspective
  3. 2025Multimodal Large Language Models for Text-rich Image Understanding: A Comprehensive Review
  4. 2024MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models
  5. 2025Seedream 2.0: A Native Chinese-English Bilingual Image Generation Foundation Model
  6. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
  7. 2024VEGA: Learning Interleaved Image-Text Comprehension in Vision-Language Large Models
  8. 2024CoMM: A Coherent Interleaved Image-Text Dataset for Multimodal Understanding and Generation