# Multi-view Consistency as Supervisory Signal for Learning Shape and Pose Prediction

arXiv: https://arxiv.org/abs/1801.03910 · HTML: https://mlsanity.com/paper/1801.03910/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jan 2018.

## Built on by (most cited first, 30 of 52)

- [Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction](https://mlsanity.com/paper/2109.00512/) (2021)
- [Unsupervised Learning of 3D Object Categories from Videos in the Wild](https://mlsanity.com/paper/2103.16552/) (2021)
- [Motion Inspired Unsupervised Perception and Prediction in Autonomous Driving](https://mlsanity.com/paper/2210.08061/) (2022)
- [Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimation](https://mlsanity.com/paper/2111.00190/) (2021)
- [UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Rendering](https://mlsanity.com/paper/2102.11870/) (2021)
- [Learning Generative Models of Textured 3D Meshes from Real-World Images](https://mlsanity.com/paper/2103.15627/) (2021)
- [SAIL-VOS 3D: A Synthetic Dataset and Baselines for Object Detection and 3D Mesh Reconstruction from Video Data](https://mlsanity.com/paper/2105.08612/) (2021)
- [LU-NeRF: Scene and Pose Estimation by Synchronizing Local Unposed NeRFs](https://mlsanity.com/paper/2306.05410/) (2023)
- [Unsupervised Learning of Fine Structure Generation for 3D Point Clouds by 2D Projection Matching](https://mlsanity.com/paper/2108.03746/) (2021)
- [Pre-train, Self-train, Distill: A simple recipe for Supersizing 3D Reconstruction](https://mlsanity.com/paper/2204.03642/) (2022)
- [Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistency](https://mlsanity.com/paper/2204.10310/) (2022)
- [Ray Priors through Reprojection: Improving Neural Radiance Fields for Novel View Extrapolation](https://mlsanity.com/paper/2205.05922/) (2022)
- [View Generalization for Single Image Textured 3D Models](https://mlsanity.com/paper/2106.06533/) (2021)
- [Self-Supervised Point Cloud Completion via Inpainting](https://mlsanity.com/paper/2111.10701/) (2021)
- [3D-RETR: End-to-End Single and Multi-View 3D Reconstruction with Transformers](https://mlsanity.com/paper/2110.08861/) (2021)
- [Fine Detailed Texture Learning for 3D Meshes with Generative Models](https://mlsanity.com/paper/2203.09362/) (2022)
- [Discovering 3D Parts from Image Collections](https://mlsanity.com/paper/2107.13629/) (2021)
- [Towards Learning Neural Representations from Shadows](https://mlsanity.com/paper/2203.15946/) (2022)
- [3D Reconstruction of Objects in Hands without Real World 3D Supervision](https://mlsanity.com/paper/2305.03036/) (2023)
- [MV-DeepSDF: Implicit Modeling with Multi-Sweep Point Clouds for 3D Vehicle Reconstruction in Autonomous Driving](https://mlsanity.com/paper/2309.16715/) (2023)
- [Progressive Learning of 3D Reconstruction Network from 2D GAN Data](https://mlsanity.com/paper/2305.11102/) (2023)
- [Mask-ToF: Learning Microlens Masks for Flying Pixel Correction in Time-of-Flight Imaging](https://mlsanity.com/paper/2103.16693/) (2021)
- [Language-driven Object Fusion into Neural Radiance Fields with Pose-Conditioned Dataset Updates](https://mlsanity.com/paper/2309.11281/) (2023)
- [Unsupervised Novel View Synthesis from a Single Image](https://mlsanity.com/paper/2102.03285/) (2021)
- [Object Wake-up: 3D Object Rigging from a Single Image](https://mlsanity.com/paper/2108.02708/) (2021)
- [An Effective Loss Function for Generating 3D Models from Single 2D Image without Rendering](https://mlsanity.com/paper/2103.03390/) (2021)
- [Lifting 2D Object Locations to 3D by Discounting LiDAR Outliers across Objects and Views](https://mlsanity.com/paper/2109.07945/) (2021)
- [WonderFree: Enhancing Novel View Quality and Cross-View Consistency for 3D Scene Exploration](https://mlsanity.com/paper/2506.20590/) (2025)
- [A Light Touch Approach to Teaching Transformers Multi-view Geometry](https://mlsanity.com/paper/2211.15107/) (2022)
- [Semi-supervised Viewpoint Estimation with Geometry-aware Conditional Generation](https://mlsanity.com/paper/2104.01103/) (2021)

## Abstract (arXiv)

We present a framework for learning single-view shape and pose prediction without using direct supervision for either. Our approach allows leveraging multi-view observations from unknown poses as supervisory signal during training. Our proposed training setup enforces geometric consistency between the independently predicted shape and pose from two views of the same instance. We consequently learn to predict shape in an emergent canonical (view-agnostic) frame along with a corresponding pose predictor. We show empirical and qualitative results using the ShapeNet dataset and observe encouragingly competitive performance to previous techniques which rely on stronger forms of supervision. We also demonstrate the applicability of our framework in a realistic setting which is beyond the scope of existing techniques: using a training dataset comprised of online product images where the underlying shape and pose are unknown.