# Can Single-View Mesh Reconstruction Generalize to Robot Camera Rotation?

arXiv: https://arxiv.org/abs/2606.22987 · HTML: https://mlsanity.com/paper/2606.22987/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 30 references that are in the map.
- First posted on arXiv 22 Jun 2026.

## Builds on (most cited first, 27 of 27)

- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [Depth Anything 3: Recovering the Visual Space from Any Views](https://mlsanity.com/paper/2511.10647/) (2025)
- [FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects](https://mlsanity.com/paper/2312.08344/) (2023)
- [Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation](https://mlsanity.com/paper/2404.15506/) (2024)
- [MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervision](https://mlsanity.com/paper/2410.19115/) (2024)
- [SAM 3D: 3Dfy Anything in Images](https://mlsanity.com/paper/2511.16624/) (2025)
- [UniDepthV2: Universal Monocular Metric Depth Estimation Made Simpler](https://mlsanity.com/paper/2502.20110/) (2025)
- [Aria Digital Twin: A New Benchmark Dataset for Egocentric 3D Machine Perception](https://mlsanity.com/paper/2306.06362/) (2023)
- [Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wild](https://mlsanity.com/paper/2207.10660/) (2022)
- [MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generation](https://mlsanity.com/paper/2412.03558/) (2024)
- [Gen3DSR: Generalizable 3D Scene Reconstruction via Divide and Conquer from a Single View](https://mlsanity.com/paper/2404.03421/) (2024)
- [SceneGen: Single-Image 3D Scene Generation in One Feedforward Pass](https://mlsanity.com/paper/2508.15769/) (2025)
- [GeoCalib: Learning Single-image Calibration with Geometric Optimization](https://mlsanity.com/paper/2409.06704/) (2024)
- [DepR: Depth Guided Single-view Scene Reconstruction with Instance-level Diffusion](https://mlsanity.com/paper/2507.22825/) (2025)
- [ShapeR: Robust Conditional 3D Shape Generation from Casual Captures](https://mlsanity.com/paper/2601.11514/) (2026)
- [SceneComplete: Open-World 3D Scene Completion in Cluttered Real World Environments for Robot Manipulation](https://mlsanity.com/paper/2410.23643/) (2024)
- [Perspective-Invariant 3D Object Detection](https://mlsanity.com/paper/2507.17665/) (2025)
- [SceneMaker: Open-set 3D Scene Generation with Decoupled De-occlusion and Pose Estimation Model](https://mlsanity.com/paper/2512.10957/) (2025)
- [Do You Know Where Your Camera Is? View-Invariant Policy Learning with Camera Conditioning](https://mlsanity.com/paper/2510.02268/) (2025)
- [Is Single-View Mesh Reconstruction Ready for Robotics?](https://mlsanity.com/paper/2505.17966/) (2025)
- [Evaluating Robustness of Monocular Depth Estimation with Procedural Scene Perturbations](https://mlsanity.com/paper/2507.00981/) (2025)
- [Picasso: Holistic Scene Reconstruction with Physics-Constrained Sampling](https://mlsanity.com/paper/2602.08058/) (2026)
- [PhysPose: Refining 6D Object Poses with Physical Constraints](https://mlsanity.com/paper/2503.23587/) (2025)
- [Real-to-Sim for Highly Cluttered Environments via Physics-Consistent Inter-Object Reasoning](https://mlsanity.com/paper/2602.12633/) (2026)
- [DICArt: Advancing Category-level Articulated Object Pose Estimation in Discrete State-Spaces](https://mlsanity.com/paper/2602.19565/) (2026)
- [Monocular Person Localization under Camera Ego-motion](https://mlsanity.com/paper/2503.02916/) (2025)
- [Toward A Better Understanding of Monocular Depth Evaluation](https://mlsanity.com/paper/2510.19814/) (2025)

## Abstract (arXiv)

Single-view mesh reconstruction predicts object meshes and spatial layouts from a single observation, making it attractive for fast robot spatial reasoning and real-to-sim digital twins. However, robot-mounted cameras naturally rotate during manipulation and navigation, while learned single-view reconstruction models often rely on view-dependent priors and may generalize poorly to out-of-distribution camera rotations. Such rotations can introduce 3D inconsistencies, incorrect layouts, and violations of physical constraints, but this failure mode remains under-evaluated. We introduce an evaluation protocol with controlled axis-wise roll, pitch, and yaw sweeps to trace errors in monocular depth estimation (MDE), canonical object meshes, camera-space layout, and physical plausibility within a representative SAM3D-style pipeline. On the Aria Digital Twin dataset and a real Franka wrist-camera sequence, camera rotations induce MDE distortion, layout drift, and collision penetration, while canonical mesh predictions remain relatively stable. A two-stage SAM3D+FoundationPose pipeline is more robust than one-stage feed-forward layout prediction, and our Gravity-Aware Refinement reduces one-stage pairwise ICP-based layout-orientation error by 47.1%. Our evaluation reveals that current single-view mesh reconstruction methods generalize poorly to robot camera rotation, and suggests that explicit gravity cues are important for reliable robotic single-view mesh reconstruction.