mlsanityData to 14 Oct 2026

RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models

arXiv 2412.07679 ↗First posted 10 Dec 2024cs.CV cs.AIcited by 89

Not on the map. Why?

Abstract (arXiv)

Agglomerative models have recently emerged as a powerful approach to training vision foundation models, leveraging multi-teacher distillation from existing models such as CLIP, DINO, and SAM. This strategy enables the efficient creation of robust models, combining the strengths of individual teachers while significantly reducing computational and resource demands. In this paper, we thoroughly analyze state-of-the-art agglomerative models, identifying critical challenges including resolution mode shifts, teacher imbalance, idiosyncratic teacher artifacts, and an excessive number of output tokens. To address these issues, we propose several novel solutions: multi-resolution training, mosaic augmentation, and improved balancing of teacher loss functions. Specifically, in the context of Vision Language Models, we introduce a token compression technique to maintain high-resolution information within a fixed token count. We release our top-performing variants at multiple scales (-B, -L, -H, and -g), along with inference code and pretrained weights

Key facts. Cited by 89 papers in the map (data through 14 Oct 2026). Builds on 45 of its 52 references that are in the map. First posted on arXiv 10 Dec 2024.

Builds on

45 of 52 references.

References · 45

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2023Segment Anythingcited by 8,234
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  6. 2016Layer Normalizationcited by 5,669
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  8. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  9. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  10. 2021Vision Transformers for Dense Predictioncited by 1,803
  11. 2019Towards VQA Models That Can Readcited by 1,718
  12. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
33 more
  1. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  2. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  3. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,069
  4. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  5. 2022Token Merging: Your ViT But Fastercited by 1,021
  6. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  7. 2023Vision Transformers Need Registerscited by 871
  8. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  9. 2023VILA: On Pre-training for Visual Language Modelscited by 692
  10. 2023DataComp: In search of the next generation of multimodal datasetscited by 644
  11. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  12. 2021InfographicVQAcited by 437
  13. 2023Data Filtering Networkscited by 250
  14. 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 229
  15. 2023Token Merging for Fast Stable Diffusioncited by 224
  16. 2024Probing the 3D Awareness of Visual Foundation Modelscited by 177
  17. 2023EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attentioncited by 172
  18. 2023AM-RADIO: Agglomerative Vision Foundation Model -- Reduce All Domains Into Onecited by 169
  19. 2022FlexiViT: One Model for All Patch Sizescited by 152
  20. 2024Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoderscited by 139
  21. 2023SAM-CLIP: Merging Vision Foundation Models towards Semantic and Spatial Understandingcited by 111
  22. 2023Region-Aware Pretraining for Open-Vocabulary Object Detection with Vision Transformerscited by 91
  23. 2024LLM Pruning and Distillation in Practice: The Minitron Approachcited by 82
  24. 2024Vision-Flan: Scaling Human-Labeled Tasks in Visual Instruction Tuningcited by 73
  25. 2024BRAVE: Broadening the visual encoding of vision-language modelscited by 69
  26. 2024Theia: Distilling Diverse Vision Foundation Models for Robot Learningcited by 61
  27. 2024Multi-Task Dense Prediction via Mixture of Low-Rank Expertscited by 58
  28. 2023NAVI: Category-Agnostic Image Collections with High-Quality 3D Shape and Pose Annotationscited by 53
  29. 2024Dense Connector for MLLMscited by 49
  30. 2024UNIC: Universal Classification Models via Multi-teacher Distillationcited by 27
  31. 2024PHI-S: Distribution Balancing for Label-Free Multi-Teacher Distillationcited by 21
  32. 2024UNIT: Unifying Image and Text Recognition in One Vision Encodercited by 12
  33. 2024VideoSAM: Open-World Video Segmentationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 40; 2026: 48.

89 papers.

Sort:

By area:

  1. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 326
  2. 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 314
  3. 2025What matters for Representation Alignment: Global Information or Spatial Structure?cited by 84
  4. 2025Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Modelscited by 70
  5. 2024DINO-Foresight: Looking into the Future with DINOcited by 45
  6. 2025Visual Representation Alignment for Multimodal Large Language Modelscited by 41
  7. 2025DUNE: Distilling a Universal Encoder from Heterogeneous 2D and 3D Teacherscited by 32
  8. 2026Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligencecited by 30
  9. 2026LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decodingcited by 28
  10. 2025Franca: Nested Matryoshka Clustering for Scalable Visual Representation Learningcited by 21
  11. 2025Thinking with Camera: A Unified Multimodal Model for Camera-Centric Understanding and Generationcited by 19
  12. 2025NVIDIA Nemotron Nano V2 VLcited by 17
38 more
  1. 2025Native-Resolution Image Synthesiscited by 15
  2. 2025SparseVILA: Decoupling Visual Sparsity for Efficient VLM Inferencecited by 15
  3. 2025From Seeing to Experiencing: Scaling Navigation Foundation Models with Reinforcement Learningcited by 12
  4. 2026C-RADIOv4 (Tech Report)cited by 12
  5. 2026INSID3: Training-Free In-Context Segmentation with DINOv3cited by 12
  6. 2026Representation Distribution Matching for One-Step Visual Generationcited by 11
  7. 2025UniUGG: Unified 3D Understanding and Generation via Geometric-Semantic Encodingcited by 10
  8. 2025Chorus: Multi-Teacher Pretraining for Holistic 3D Gaussian Scene Encodingcited by 10
  9. 2026Efficient Universal Perception Encodercited by 10
  10. 2025AnyTask: an Automated Task and Data Generation Framework for Advancing Sim-to-Real Policy Learningcited by 8
  11. 2025NAF: Zero-Shot Feature Upsampling via Neighborhood Attention Filteringcited by 6
  12. 2025Pathryoshka: Compressing Pathology Foundation Models via Multi-Teacher Knowledge Distillation with Nested Embeddingscited by 6
  13. 2026Diffusing in the Right Space: A Systematic Study of Latent Diffusabilitycited by 6
  14. 2025RADSeg: Unleashing Parameter and Compute Efficient Zero-Shot Open-Vocabulary Segmentation Using Agglomerative Modelscited by 5
  15. 2026OpenVision 3: A Family of Unified Visual Encoder for Both Understanding and Generationcited by 5
  16. 2026What Matters for Scalable and Robust Learning in End-to-End Driving Planners?cited by 5
  17. 2025Schrodinger Audio-Visual Editor: Object-Level Audiovisual Removalcited by 4
  18. 2026MonkeyOCRv2: A Visual-Text Foundation Model for Document AIcited by 4
  19. 2025Fair Foundation Models for Medical Image Analysis: Challenges and Perspectivescited by 3
  20. 2025Agglomerating Large Vision Encoders via Distillation for VFSS Segmentationcited by 3
  21. 2025FORLA: Federated Object-centric Representation Learning with Slot Attentioncited by 3
  22. 2025DIP: Unsupervised Dense In-Context Post-training of Visual Representationscited by 3
  23. 2025Fantastic Features and Where to Find Them: A Probing Method to combine Features from Multiple Foundation Modelscited by 3
  24. 2025SigLino: Efficient Multi-Teacher Distillation for Agglomerative Vision Foundation Modelscited by 3
  25. 2026DeFM: Learning Foundation Representations from Depth for Roboticscited by 3
  26. 2026WildOS: Open-Vocabulary Object Search in the Wildcited by 3
  27. 2025GaussianCross: Cross-modal Self-supervised 3D Representation Learning via Gaussian Splattingcited by 2
  28. 2025Image Recognition with Vision and Language Embeddings of VLMscited by 2
  29. 2025Beyond 'Templates': Category-Agnostic Object Pose, Size, and Shape Estimation from a Single Viewcited by 2
  30. 20253D-Aware Multi-Task Learning with Cross-View Correlations for Dense Scene Understandingcited by 2
  31. 2026Brewing Stronger Features: Dual-Teacher Distillation for Multispectral Earth Observationcited by 2
  32. 2026Scaling Dense Event-Stream Pretraining from Visual Foundation Modelscited by 2
  33. 2026On the Generalization Capabilities, Design Choices and Limitations of Keypoint Imitation Learningcited by 2
  34. 2026DVSM: Decoder-only View Synthesis Model Done Rightcited by 2
  35. 2026FiberTune: Preserving Action-Fiber Visual Residuals in Vision-Language-Action Fine-Tuningcited by 2
  36. 2025UINO-FSS: Unifying Representation Learning and Few-shot Segmentation via Hierarchical Distillation and Mamba-HyperCorrelationcited by 1
  37. 2025DynaRend: Learning 3D Dynamics via Masked Future Rendering for Robotic Manipulationcited by 1
  38. 2025NVIDIA Nemotron Parse 1.1cited by 1

50 most cited of 89 All 89 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026The Professor: Multi-Teacher Unsupervised Prompt Distillation for Vision-Language Models
  2. 2023Distilling Large Vision-Language Model with Out-of-Distribution Generalizability
  3. 2024ALLaVA: Harnessing GPT4V-Synthesized Data for Lite Vision-Language Models
  4. 2024POINTS: Improving Your Vision-language Model with Affordable Strategies
  5. 2026DataComp-VLM: Improved Open Datasets for Vision-Language Models
  6. 202620/20 Vision Language Models: A Prescription for Better VLMs through Data Curation Alone
  7. 2026Underrepresented in Foundation Model Pretraining Data? A One-Shot Probe
  8. 2023Robust Fine-Tuning of Vision-Language Models for Domain Generalization