OmniPoint: Universal Monocular Metric Pointcloud from Any Camera
Not on the map. Why?
Abstract (arXiv)
Recovering metric 3D geometry from monocular images is a fundamental computer vision task, yet current methods remain heavily fragmented by fixed camera model assumptions and inflexible input schemes. We present OmniPoint, a unified framework designed to generalize metric reconstruction across diverse imaging sensors, including pinhole, fisheye, and equirectangular projections, while accommodating varying geometric priors. To overcome projection rigidity, OmniPoint abandons conventional planar depth regression. It instead adopts a decoupled ray and distance representation alongside a decoupled training objective, explicitly separating the camera projection model from the scene structure. To address the severe scarcity of training data for alternative cameras, we introduce a bidirectional augmentation strategy that explicitly bridges labeled perspective data and unlabeled omnidirectional domains in 3D space. Furthermore, to seamlessly integrate optional inputs like camera intrinsics or sparse depth without destabilizing the network through feature distribution shifts, we propose a robust information injection mechanism. This mechanism utilizes learnable input state embeddings to resolve architectural ambiguity and applies vectorized Gaussian smoothing to densify irregular measurements. Extensive experiments demonstrate that OmniPoint achieves state-of-the-art zero-shot performance across multiple benchmarks, establishing a robust new standard for unified monocular 3D reconstruction.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 59 of its 67 references that are in the map. First posted on arXiv 8 Sep 2026.
Builds on
59 of 67 references.
References · 59
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
- 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
- 2021Vision Transformers for Dense Predictioncited by 1,803
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
- 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
- 2024Depth Anything V2cited by 1,422
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
- 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
- 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
38 more
- 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
- 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 657
- 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 612
- 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 604
- 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 575
- 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 506
- 2019BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networkscited by 490
- 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 454
- 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 406
- 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 404
- 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 395
- 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 382
- 2020Virtual KITTI 2cited by 353
- 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 327
- 2023OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generationcited by 326
- 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 299
- 2019Structured3D: A Large Photo-realistic Dataset for Structured 3D Modelingcited by 286
- 2018DeepMVS: Learning Multi-view Stereopsiscited by 277
- 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 276
- 2025MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Detailscited by 261
- 2019DIODE: A Dense Indoor and Outdoor DEpth Datasetcited by 240
- 2023BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motioncited by 236
- 2025UniDepthV2: Universal Monocular Metric Depth Estimation Made Simplercited by 173
- 2023Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereocited by 172
- 2023DynamicStereo: Consistent Dynamic Depth from Stereo Videoscited by 148
- 2018Evaluation of CNN-based Single-Image Depth Estimation Methodscited by 138
- 20193D Ken Burns Effect from a Single Imagecited by 128
- 2021SMD-Nets: Stereo Mixture Density Networkscited by 120
- 2024RGBD Objects in the Wild: Scaling Real-World 3D Object Learning from RGB-D Videoscited by 115
- 2018Synscapes: A Photorealistic Synthetic Dataset for Street Scene Parsingcited by 106
- 2024Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimationcited by 74
- 2025UniK3D: Universal Camera Monocular 3D Estimationcited by 62
- 2025OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modelingcited by 62
- 2024Bilateral Propagation Network for Depth Completioncited by 60
- 2023All for One, and One for All: UrbanSyn Dataset, the third Musketeer of Synthetic Driving Scenescited by 41
- 2018Self-Supervised Learning of Depth and Camera Motion from 360deg Videoscited by 40
- 2025Depth Any Camera: Zero-Shot Metric Depth Estimation from Any Cameracited by 40
50 most cited of 59 All 59 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.