Can Single-View Mesh Reconstruction Generalize to Robot Camera Rotation?
Not on the map. Why?
Abstract (arXiv)
Single-view mesh reconstruction predicts object meshes and spatial layouts from a single observation, making it attractive for fast robot spatial reasoning and real-to-sim digital twins. However, robot-mounted cameras naturally rotate during manipulation and navigation, while learned single-view reconstruction models often rely on view-dependent priors and may generalize poorly to out-of-distribution camera rotations. Such rotations can introduce 3D inconsistencies, incorrect layouts, and violations of physical constraints, but this failure mode remains under-evaluated. We introduce an evaluation protocol with controlled axis-wise roll, pitch, and yaw sweeps to trace errors in monocular depth estimation (MDE), canonical object meshes, camera-space layout, and physical plausibility within a representative SAM3D-style pipeline. On the Aria Digital Twin dataset and a real Franka wrist-camera sequence, camera rotations induce MDE distortion, layout drift, and collision penetration, while canonical mesh predictions remain relatively stable. A two-stage SAM3D+FoundationPose pipeline is more robust than one-stage feed-forward layout prediction, and our Gravity-Aware Refinement reduces one-stage pairwise ICP-based layout-orientation error by 47.1%. Our evaluation reveals that current single-view mesh reconstruction methods generalize poorly to robot camera rotation, and suggests that explicit gravity cues are important for reliable robotic single-view mesh reconstruction.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 30 references that are in the map. First posted on arXiv 22 Jun 2026.
Builds on
27 of 30 references.
References · 27
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
- 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 411
- 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 389
- 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 290
- 2025SAM 3D: 3Dfy Anything in Imagescited by 230
- 2025UniDepthV2: Universal Monocular Metric Depth Estimation Made Simplercited by 170
- 2023Aria Digital Twin: A New Benchmark Dataset for Egocentric 3D Machine Perceptioncited by 166
- 2022Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wildcited by 159
- 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generationcited by 113
- 2024Gen3DSR: Generalizable 3D Scene Reconstruction via Divide and Conquer from a Single Viewcited by 59
- 2024GeoCalib: Learning Single-image Calibration with Geometric Optimizationcited by 50
15 more
- 2025SceneGen: Single-Image 3D Scene Generation in One Feedforward Passcited by 49
- 2025DepR: Depth Guided Single-view Scene Reconstruction with Instance-level Diffusioncited by 23
- 2026ShapeR: Robust Conditional 3D Shape Generation from Casual Capturescited by 19
- 2024SceneComplete: Open-World 3D Scene Completion in Cluttered Real World Environments for Robot Manipulationcited by 17
- 2025Perspective-Invariant 3D Object Detectioncited by 16
- 2025SceneMaker: Open-set 3D Scene Generation with Decoupled De-occlusion and Pose Estimation Modelcited by 15
- 2025Do You Know Where Your Camera Is? View-Invariant Policy Learning with Camera Conditioningcited by 14
- 2025Is Single-View Mesh Reconstruction Ready for Robotics?cited by 6
- 2025Evaluating Robustness of Monocular Depth Estimation with Procedural Scene Perturbationscited by 4
- 2026Picasso: Holistic Scene Reconstruction with Physics-Constrained Samplingcited by 4
- 2025PhysPose: Refining 6D Object Poses with Physical Constraintscited by 3
- 2026Real-to-Sim for Highly Cluttered Environments via Physics-Consistent Inter-Object Reasoningcited by 3
- 2026DICArt: Advancing Category-level Articulated Object Pose Estimation in Discrete State-Spacescited by 2
- 2025Monocular Person Localization under Camera Ego-motioncited by 1
- 2025Toward A Better Understanding of Monocular Depth Evaluationcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026GeneralVLA-2: Geometry-Aware Reconstruction and Governed Memory for Robot Planning
- 2026Robo3R: Enhancing Robotic Manipulation with Accurate Feed-Forward 3D Reconstruction
- 2024Exploiting Priors from 3D Diffusion Models for RGB-Based One-Shot View Planning
- 2026OC-VLA++: Monocular Geometry-Guided Cross-View Consistency for Viewpoint-Robust Robotic Manipulation
- 2026RoGSW4RLD: Feed-Forward 4D Gaussian Lifting for Robot World Model Rollouts
- 2025UnPose: Uncertainty-Guided Diffusion Priors for Zero-Shot Pose Estimation
- 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization
- 2025ACT-R: Adaptive Camera Trajectories for Single View 3D Reconstruction