mlsanityData to 14 Oct 2026

DINOcular: Self-Supervised Visuospatial Representations

arXiv 2608.27226 ↗First posted 27 Aug 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce a self-supervised framework for learning joint visuospatial representations from RGB-D observations. While modern vision foundation models are trained almost exclusively on RGB images, many embodied systems have access to explicit depth sensing, which provides geometric information that monocular inputs cannot recover. Our method integrates depth-derived geometric priors with a visual backbone through inter-patch and intra-patch fusion, enabling the model to encode both appearance and spatial structure efficiently. The resulting representation shows promising improvements on 3D awareness while preserving semantic transfer: it outperforms prior methods of comparable scale on multiple 3D geometry benchmarks, and remains competitive when probed for standard RGB-D semantic segmentation tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 47 of its 54 references that are in the map. First posted on arXiv 27 Aug 2026.

Builds on

47 of 54 references.

References · 47

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  7. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  8. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  9. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  10. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  11. 2025DINOv3cited by 1,414
  12. 2024Grounding Image Matching in 3D with MASt3Rcited by 983
35 more
  1. 2025SAM 3: Segment Anything with Conceptscited by 906
  2. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 903
  3. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
  4. 2020AdaBins: Depth Estimation using Adaptive Binscited by 601
  5. 2025Continuous 3D Perception Model with Persistent Statecited by 548
  6. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 528
  7. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 442
  8. 2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 291
  9. 2024Probing the 3D Awareness of Visual Foundation Modelscited by 177
  10. 2025FoundationStereo: Zero-Shot Stereo Matchingcited by 173
  11. 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 164
  12. 2022CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completioncited by 160
  13. 2022Multimodal Token Fusion for Vision Transformerscited by 136
  14. 2020Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentationcited by 122
  15. 2018Spreading vectors for similarity searchcited by 118
  16. 2023OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD Modelscited by 115
  17. 2025Sonata: Self-Supervised Learning of Reliable Point Representationscited by 110
  18. 2019ACNet: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentationcited by 107
  19. 2020Smooth-AP: Smoothing the Path Towards Large-Scale Image Retrievalcited by 98
  20. 2023RMT: Retentive Networks Meet Vision Transformerscited by 68
  21. 2023DFormer: Rethinking RGBD Representation Learning for Semantic Segmentationcited by 60
  22. 2023NAVI: Category-Agnostic Image Collections with High-Quality 3D Shape and Pose Annotationscited by 53
  23. 2024Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene Understandingcited by 49
  24. 2025Concerto: Joint 2D-3D Self-Supervised Learning Emerges Spatial Representationscited by 45
  25. 2025DUNE: Distilling a Universal Encoder from Heterogeneous 2D and 3D Teacherscited by 32
  26. 2025DFormerv2: Geometry Self-Attention for RGBD Semantic Segmentationcited by 31
  27. 2025Simplifying DINO via Coding Rate Regularizationcited by 21
  28. 2025Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenescited by 20
  29. 2025Learning the RoPEs: Better 2D and 3D Position Encodings with STRINGcited by 19
  30. 2024Multiview Equivariance Improves 3D Correspondence Understanding with Minimal Feature Finetuningcited by 14
  31. 2022Early or Late Fusion Matters: Efficient RGB-D Fusion in Vision Transformers for 3D Object Recognitioncited by 12
  32. 2025Unified Semantic Transformer for 3D Scene Understandingcited by 8
  33. 2025DiffPixelFormer: Differential Pixel-Aware Transformer for RGB-D Indoor Scene Segmentationcited by 2
  34. 20263D-IDE: 3D Implicit Depth Emergentcited by 1
  35. 2026GeomPrompt: Geometric Prompt Learning for RGB-D Semantic Segmentation Under Missing and Degraded Depthcited by 1

Built on by

Not cited yet.