mlsanityData to 7 Oct 2026

Understanding Multi-View Transformers

arXiv 2510.24907 ↗First posted 28 Oct 2025cs.CV cs.AI cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Multi-view transformers such as DUSt3R are revolutionizing 3D vision by solving 3D tasks in a feed-forward manner. However, contrary to previous optimization-based pipelines, the inner mechanisms of multi-view transformers are unclear. Their black-box nature makes further improvements beyond data scaling challenging and complicates usage in safety- and reliability-critical applications. Here, we present an approach for probing and visualizing 3D representations from the residual connections of the multi-view transformers' layers. In this manner, we investigate a variant of the DUSt3R model, shedding light on the development of its latent state across blocks, the role of the individual layers, and suggest how it differs from methods with stronger inductive biases of explicit global pose. Finally, we show that the investigated variant of DUSt3R estimates correspondences that are refined with reconstructed geometry. The code used for the analysis is available at https://github.com/JulienGaubil/und3rstand .

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 51 of its 61 references that are in the map. First posted on arXiv 28 Oct 2025.

Builds on

51 of 61 references.

References · 51

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  5. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  6. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  7. 2022Mass-Editing Memory in a Transformercited by 976
  8. 2024Grounding Image Matching in 3D with MASt3Rcited by 960
  9. 2023Vision Transformers Need Registerscited by 844
  10. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  11. 2022Transformers learn in-context by gradient descentcited by 714
  12. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
38 more
  1. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  2. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  3. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  4. 2023Emergent Correspondence from Image Diffusioncited by 437
  5. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 431
  6. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  7. 2014Understanding Deep Image Representations by Inverting Themcited by 399
  8. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
  9. 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 356
  10. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  11. 2019A Multiscale Visualization of Attention in the Transformer Modelcited by 292
  12. 20243D Reconstruction with Spatial Memorycited by 269
  13. 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 254
  14. 2024No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Imagescited by 241
  15. 2023Visual Geometry Grounded Deep Structure From Motioncited by 225
  16. 2024Global Structure-from-Motion Revisitedcited by 215
  17. 2024MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priorscited by 198
  18. 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 189
  19. 2024Probing the 3D Awareness of Visual Foundation Modelscited by 175
  20. 2023The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networkscited by 153
  21. 2025MUSt3R: Multi-view Network for Stereo 3D Reconstructioncited by 139
  22. 2022Analyzing Transformers in Embedding Spacecited by 131
  23. 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videoscited by 89
  24. 2017Residual Connections Encourage Iterative Inferencecited by 74
  25. 2024Scene Coordinate Reconstruction: Posing of Image Collections via Incremental Learning of a Relocalizercited by 73
  26. 2024Large Spatial Model: End-to-end Unposed Images to Semantic 3Dcited by 73
  27. 2023Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIPcited by 62
  28. 2024FlowMap: High-Quality Camera Poses, Intrinsics, and Depth via Gradient Descentcited by 61
  29. 2024Understanding Information Storage and Transfer in Multi-modal Large Language Modelscited by 48
  30. 2024Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene Understandingcited by 46
  31. 2024Interpreting the Second-Order Effects of Neurons in CLIPcited by 36
  32. 2023A General Protocol to Probe Large Vision Models for 3D Physical Understandingcited by 32
  33. 2022Visualizing and Understanding Patch Interactions in Vision Transformercited by 26
  34. 2019On the Units of GANs (Extended Abstract)cited by 19
  35. 2024Decomposing and Interpreting Image Representations via Text in ViTs Beyond CLIPcited by 19
  36. 2024Residual Stream Analysis with Multi-Layer SAEscited by 18
  37. 2023Scale Alone Does not Improve Mechanistic Interpretability in Vision Modelscited by 17
  38. 2024Multiview Equivariance Improves 3D Correspondence Understanding with Minimal Feature Finetuningcited by 13

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026VGGT-SLAM 2.0: Real-time Dense Feed-forward Scene Reconstructioncited by 22
  2. 2026D'ej`a View: Looping Transformers for Multi-View 3D Reconstructioncited by 4
  3. 2026Distill3R: A Pipeline for Democratizing 3D Foundation Models on Commodity Hardwarecited by 3
  4. 2026Pano3D: Unified 3D Reconstruction and Panoptic Segmentationcited by 0
  5. 2026What VGGT Knows About Overlap: Probing Geometric Foundation Models for Co-Visibilitycited by 0
  6. 2026RoMa-Omega: What Feed-Forward 3D Models Know About Image Matchingcited by 0
  7. 2026OTT3R: Multi-View 3D Reconstruction and Fast Dataset Generation at 1% Computecited by 0
  8. 2026Emergent Multi-View Geometry Through Self-Distillationcited by 0