mlsanityData to 14 Oct 2026

Multi-View Foundation Models

arXiv 2512.15708 ↗First posted 17 Dec 2025cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Foundation models are vital tools in various Computer Vision applications. They take as input a single RGB image and output a deep feature representation that is useful for various applications. However, in case we have multiple views of the same 3D scene, they operate on each image independently and do not always produce consistent features for the same 3D point. We propose a way to convert a Foundation Model into a Multi-View Foundation Model. Such a model takes as input a set of images and outputs a feature map for each image such that the features of corresponding points are as consistent as possible. This approach bypasses the need to build a consistent 3D model of the features and allows direct manipulation in the image space. Specifically, we show how to augment Transformers-based foundation models (i.e., DINO, SAM, CLIP) with intermediate 3D-aware attention layers that help match features across different views. As leading examples, we show surface normal estimation and multi-view segmentation tasks. Quantitative experiments show that our method improves feature matching considerably compared to current foundation models.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 44 of its 52 references that are in the map. First posted on arXiv 17 Dec 2025.

Builds on

44 of 52 references.

References · 44

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023Segment Anythingcited by 8,234
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  6. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  7. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  8. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  9. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  10. 2022Video Diffusion Modelscited by 2,305
  11. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  12. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
32 more
  1. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,613
  2. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  3. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  4. 2025DINOv3cited by 1,414
  5. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
  6. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 964
  7. 2023Vision Transformers Need Registerscited by 871
  8. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
  9. 2021In-Place Scene Labelling and Understanding with Implicit Scene Representationcited by 415
  10. 2023Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fieldscited by 364
  11. 2022Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentationcited by 344
  12. 2021Deep ViT Features as Dense Visual Descriptorscited by 330
  13. 2025π3: Permutation-Equivariant Visual Geometry Learningcited by 319
  14. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 274
  15. 2024Probing the 3D Awareness of Visual Foundation Modelscited by 177
  16. 2022Panoptic Lifting for 3D Scene Understanding with Neural Fieldscited by 174
  17. 2023The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimationcited by 116
  18. 2023HoloDiffusion: Training a 3D Diffusion Model using 2D Imagescited by 106
  19. 2020Smooth-AP: Smoothing the Path Towards Large-Scale Image Retrievalcited by 98
  20. 2023Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Trainingcited by 86
  21. 2023Telling Left from Right: Identifying Geometry-Aware Semantic Correspondencecited by 86
  22. 2023Nerfbusters: Removing Ghostly Artifacts from Casually Captured NeRFscited by 76
  23. 2024Improving 2D Feature Representations by 3D-Aware Fine-Tuningcited by 71
  24. 2024DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Videocited by 64
  25. 2023Masked Scene Contrast: A Scalable Framework for Unsupervised 3D Representation Learningcited by 56
  26. 2023NAVI: Category-Agnostic Image Collections with High-Quality 3D Shape and Pose Annotationscited by 53
  27. 2023FeatureNeRF: Learning Generalizable NeRFs by Distilling Foundation Modelscited by 47
  28. 2022nerf2nerf: Pairwise Registration of Neural Radiance Fieldscited by 39
  29. 2024Lift3D: Zero-Shot Lifting of Any 2D Vision Model to 3Dcited by 15
  30. 2025Exploring Temporally-Aware Features for Point Trackingcited by 15
  31. 2024Multiview Equivariance Improves 3D Correspondence Understanding with Minimal Feature Finetuningcited by 14
  32. 2024Towards Foundation Models for 3D Vision: How Close Are We?cited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Geometric Consistency Protocol for Foundation Model Features in Multi-View Satellite Imagerycited by 1
  2. 2026Emergent Multi-View Geometry Through Self-Distillationcited by 0