mlsanityData to 14 Oct 2026

RADIO1D: Elastic Representations for Condensed Vision Modeling

arXiv 2607.03624 ↗First posted 3 Jul 2026cs.CV cs.AI cs.LGcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

This paper challenges the assumption that vision-language models (VLMs) require fixed patch-based 2D vision features. Analyzing fine-tuned vision encoders, we find that representations become increasingly abstract and less spatially coherent during VLM training. Notably, models trained with image-text alignment (such as SigLIP2) develop a small number of specialized tokens that effectively summarize global image content. Building on this, we introduce RADIO1D, which compresses images into a compact, variable-length 1D token sequence using multi-teacher knowledge distillation and an autoencoder design. The resulting representations exhibit strong hierarchical summarization, enabling accurate scene understanding - even with a single token - and support improved composition-aware image retrieval. In VLMs, RADIO1D provides flexible accuracy-efficiency tradeoffs through adjustable token counts, delivering competitive performance on diverse multimodal benchmarks with lower computational overhead and better accuracy.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 39 of its 48 references that are in the map. First posted on arXiv 3 Jul 2026.

Builds on

39 of 48 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024What matters when building vision-language models?3 May
  2. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond24 Aug

Other references · 37

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  8. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  9. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  10. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  11. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  12. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
25 more
  1. 2019Towards VQA Models That Can Readcited by 1,718
  2. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  3. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  4. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  5. 2022Token Merging: Your ViT But Fastercited by 1,021
  6. 2023Vision Transformers Need Registerscited by 871
  7. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  8. 2016A Diagram Is Worth A Dozen Imagescited by 804
  9. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
  10. 2023DataComp: In search of the next generation of multimodal datasetscited by 644
  11. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  12. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  13. 2021InfographicVQAcited by 437
  14. 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 302
  15. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 295
  16. 2015Net2Net: Accelerating Learning via Knowledge Transfercited by 250
  17. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoningcited by 173
  18. 2023AM-RADIO: Agglomerative Vision Foundation Model -- Reduce All Domains Into Onecited by 169
  19. 2024Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoderscited by 139
  20. 2025FlexTok: Resampling Images into 1D Token Sequences of Flexible Lengthcited by 110
  21. 2025Beyond Attention or Similarity: Maximizing Conditional Diversity for Token Pruning in MLLMscited by 96
  22. 2023Getting ViT in Shape: Scaling Laws for Compute-Optimal Model Designcited by 89
  23. 2024RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Modelscited by 89
  24. 2024LLaVA-UHD v2: an MLLM Integrating High-Resolution Semantic Pyramid via Hierarchical Window Transformercited by 8
  25. 2025Benchmarking DINOv3 for Multi-Task Stroke Analysis on Non-Contrast CTcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026VisionWeave: Weaving Elastic Visual Representations as a Native Capability of MLLMs
  2. 2026From Pixels to Words -- Towards Native One-Vision Models at Scale
  3. 2025From Pixels to Words -- Towards Native Vision-Language Primitives at Scale
  4. 2025Reading Images Like Texts: Sequential Image Understanding in Vision-Language Models
  5. 2025Growing Visual Generative Capacity for Pre-Trained MLLMs
  6. 2025Patch-as-Decodable-Token: Towards Unified Multi-Modal Vision Tasks in MLLMs
  7. 2026The Spatial Blindspot of Vision-Language Models
  8. 2025MMLongBench: Benchmarking Long-Context Vision-Language Models Effectively and Thoroughly