Efficient Architectures for High Resolution Vision-Language Models
On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning
Abstract (arXiv)
Vision-Language Models (VLMs) have recently experienced significant advancements. However, challenges persist in the accurate recognition of fine details within high resolution images, which limits performance in multiple tasks. This work introduces Pheye, a novel architecture that efficiently processes high-resolution images while training fewer parameters than similarly sized VLMs. Notably, Pheye achieves a high efficiency while maintaining strong performance, particularly in tasks that demand fine-grained image understanding and/or the handling of scene-text.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 29 of its 44 references that are in the map. First posted on arXiv 5 Jan 2025.
Builds on
29 of 44 references.
Closest ancestors · 3
Same direction, newest first
- 2024MobileVLM V2: Faster and Stronger Baseline for Vision Language Model6 Feb
- 2023PaLI-3 Vision Language Models: Smaller, Faster, Stronger13 Oct
- 2022Flamingo: a Visual Language Model for Few-Shot Learning29 Apr
Other references · 26
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023GPT-4 Technical Reportcited by 18,132
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2023Segment Anythingcited by 8,158
- 2016Layer Normalizationcited by 5,579
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
- 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
- 2015VQA: Visual Question Answeringcited by 2,489
- 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
- 2019Towards VQA Models That Can Readcited by 1,696
- 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
14 more
- 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
- 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
- 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
- 2018nocaps: novel object captioning at scalecited by 444
- 2020TextCaps: a Dataset for Image Captioning with Reading Comprehensioncited by 398
- 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Modelscited by 356
- 2019Scene Text Visual Question Answeringcited by 327
- 2023LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understandingcited by 272
- 2023MIMIC-IT: Multi-Modal In-Context Instruction Tuningcited by 264
- 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 226
- 2023SVIT: Scaling up Visual Instruction Tuningcited by 161
- 2023To See is to Believe: Prompting GPT-4V for Better Visual Instruction Tuningcited by 143
- 2024DOCCI: Descriptions of Connected and Contrasting Imagescited by 130
- 2022SmallCap: Lightweight Image Captioning Prompted with Retrieval Augmentationcited by 82
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025A Survey on Efficient Vision-Language Models
- 2025Native Visual Understanding: Resolving Resolution Dilemmas in Vision-Language Models
- 2024FastVLM: Efficient Vision Encoding for Vision Language Models
- 2024What matters when building vision-language models?
- 2025Flash-VL 2B: Optimizing Vision-Language Model Performance for Ultra-Low Latency and High Throughput
- 2024DeepSeek-VL: Towards Real-World Vision-Language Understanding
- 2025jina-vlm: Small Multilingual Vision Language Model
- 202620/20 Vision Language Models: A Prescription for Better VLMs through Data Curation Alone