mlsanityData to 14 Oct 2026

VARCO-VISION: Expanding Frontiers in Korean Vision-Language Models

arXiv 2411.19103 ↗First posted 28 Nov 2024cs.CV cs.CLcited by 6

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

In this paper, we introduce an open-source Korean-English vision-language model (VLM), VARCO-VISION. We incorporate a step-by-step training strategy that allows a model learn both linguistic and visual information while preserving the backbone model's knowledge. Our model demonstrates outstanding performance in diverse settings requiring bilingual image-text understanding and generation abilities compared to models of similar size. VARCO-VISION is also capable of grounding, referring, and OCR, expanding its usage and potential applications for real-world scenarios. In addition to the model, we release five Korean evaluation datasets, including four closed-set and one openset benchmarks. We anticipate that our milestone will broaden the opportunities for AI researchers aiming to train VLMs. VARCO-VISION is available at https://huggingface.co/NCSOFT/VARCO-VISION-14B.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 22 of its 41 references that are in the map. First posted on arXiv 28 Nov 2024.

Builds on

22 of 41 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models25 Sep
  2. 2024Building and better understanding vision-language models: insights and future directions22 Aug
  3. 2023Kosmos-2: Grounding Multimodal Large Language Models to the World26 Jun

Other references · 19

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  3. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  4. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  5. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  6. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  7. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  8. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
  9. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
  10. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
  11. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  12. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
7 more
  1. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 355
  2. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 353
  3. 2024Pixtral 12Bcited by 179
  4. 2024Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Modelscited by 113
  5. 2024A Survey on Benchmarks of Multimodal Large Language Modelscited by 82
  6. 2024Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languagescited by 64
  7. 2024EXAONE 3.0 7.8B Instruction Tuned Language Modelcited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025KRETA: A Benchmark for Korean Reading and Reasoning in Text-Rich VQA Attuned to Diverse Visual Contextscited by 6
  2. 2025VARCO-VISION-2.0 Technical Reportcited by 6
  3. 2025KOFFVQA: An Objectively Evaluated Free-form VQA Benchmark for Large Vision-Language Models in the Korean Languagecited by 5
  4. 2025Trillion 7B Technical Reportcited by 4
  5. 2026Anthropogenic Regional Adaptation in Multimodal Vision-Language Modelcited by 2
  6. 2026HyperCLOVA X 32B Thinkcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025VLURes: Benchmarking VLM Visual and Linguistic Understanding in Low-Resource Languages
  2. 2024Heron-Bench: A Benchmark for Evaluating Vision Language Models in Japanese
  3. 2024An Introduction to Vision-Language Modeling
  4. 2024Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions
  5. 2025Exploring OCR-augmented Generation for Bilingual VQA
  6. 2025jina-vlm: Small Multilingual Vision Language Model
  7. 2022Vision-Language Intelligence: Tasks, Representation Learning, and Large Models
  8. 2026Multilingual Training and Evaluation Resources for Vision-Language Models