# Vid2CAD: CAD Model Alignment using Multi-View Constraints from Videos

arXiv: https://arxiv.org/abs/2012.04641 · HTML: https://mlsanity.com/paper/2012.04641/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Dec 2020.

## Built on by (most cited first, 30 of 33)

- [SAM 3D: 3Dfy Anything in Images](https://mlsanity.com/paper/2511.16624/) (2025)
- [Multi-level Feature Learning for Contrastive Multi-view Clustering](https://mlsanity.com/paper/2106.11193/) (2021)
- [ROCA: Robust CAD Model Retrieval and Alignment from a Single Image](https://mlsanity.com/paper/2112.01988/) (2021)
- [DiffCAD: Weakly-Supervised Probabilistic CAD Model Retrieval and Alignment from an RGB Image](https://mlsanity.com/paper/2311.18610/) (2023)
- [D3D-HOI: Dynamic 3D Human-Object Interactions from Videos](https://mlsanity.com/paper/2108.08420/) (2021)
- [ODAM: Object Detection, Association, and Mapping using Posed RGB Video](https://mlsanity.com/paper/2108.10165/) (2021)
- [OmniNOCS: A unified NOCS dataset and model for 3D lifting of 2D objects](https://mlsanity.com/paper/2407.08711/) (2024)
- [LiteReality: Graphics-Ready 3D Scene Reconstruction from RGB-D Scans](https://mlsanity.com/paper/2507.02861/) (2025)
- [Point Scene Understanding via Disentangled Instance Mesh Reconstruction](https://mlsanity.com/paper/2203.16832/) (2022)
- [SPARC: Sparse Render-and-Compare for CAD model alignment in a single RGB image](https://mlsanity.com/paper/2210.01044/) (2022)
- [RayTran: 3D pose estimation and shape reconstruction of multiple objects from videos with ray-traced transformers](https://mlsanity.com/paper/2203.13296/) (2022)
- [ObjectMatch: Robust Registration using Canonical Object Correspondences](https://mlsanity.com/paper/2212.01985/) (2022)
- [CAD-Estate: Large-scale CAD Model Annotation in RGB Videos](https://mlsanity.com/paper/2306.09011/) (2023)
- [Pixel-Aligned Recurrent Queries for Multi-View 3D Object Detection](https://mlsanity.com/paper/2310.01401/) (2023)
- [Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modeling](https://mlsanity.com/paper/2411.19492/) (2024)
- [ELLIPSDF: Joint Object Pose and Shape Optimization with a Bi-level Ellipsoid and Signed Distance Function Description](https://mlsanity.com/paper/2108.00355/) (2021)
- [Uncertainty-aware 3D Object-Level Mapping with Deep Shape Priors](https://mlsanity.com/paper/2309.09118/) (2023)
- [FastCAD: Real-Time CAD Retrieval and Alignment from Scans and Videos](https://mlsanity.com/paper/2403.15161/) (2024)
- [Estimating Generic 3D Room Structures from 2D Annotations](https://mlsanity.com/paper/2306.09077/) (2023)
- [CORSAIR: Convolutional Object Retrieval and Symmetry-AIded Registration](https://mlsanity.com/paper/2103.06911/) (2021)
- [LASA: Instance Reconstruction from Real Scans using A Large-scale Aligned Shape Annotation Dataset](https://mlsanity.com/paper/2312.12418/) (2023)
- [Learning 3D Object Spatial Relationships from Pre-trained 2D Diffusion Models](https://mlsanity.com/paper/2503.19914/) (2025)
- [Leveraging Geometry for Shape Estimation from a Single RGB Image](https://mlsanity.com/paper/2111.05615/) (2021)
- [Sparse Multi-Object Render-and-Compare](https://mlsanity.com/paper/2310.11184/) (2023)
- [CropCraft: Complete Structural Characterization of Crop Plants From Images](https://mlsanity.com/paper/2411.09693/) (2024)
- [Active 6D Pose Estimation for Textureless Objects using Multi-View RGB Frames](https://mlsanity.com/paper/2503.03726/) (2025)
- [Weakly-Supervised End-to-End CAD Retrieval to Scan Objects](https://mlsanity.com/paper/2203.12873/) (2022)
- [TransPoser: Transformer as an Optimizer for Joint Object Shape and Pose Estimation](https://mlsanity.com/paper/2303.13477/) (2023)
- [PLOT: Pseudo-Labeling via Object Tracking for Monocular 3D Object Detection](https://mlsanity.com/paper/2507.02393/) (2025)
- [Zero-shot Inexact CAD Model Alignment from a Single Image](https://mlsanity.com/paper/2507.03292/) (2025)

## Abstract (arXiv)

We address the task of aligning CAD models to a video sequence of a complex scene containing multiple objects. Our method can process arbitrary videos and fully automatically recover the 9 DoF pose for each object appearing in it, thus aligning them in a common 3D coordinate frame. The core idea of our method is to integrate neural network predictions from individual frames with a temporally global, multi-view constraint optimization formulation. This integration process resolves the scale and depth ambiguities in the per-frame predictions, and generally improves the estimate of all pose parameters. By leveraging multi-view constraints, our method also resolves occlusions and handles objects that are out of view in individual frames, thus reconstructing all objects into a single globally consistent CAD representation of the scene. In comparison to the state-of-the-art single-frame method Mask2CAD that we build on, we achieve substantial improvements on the Scan2CAD dataset (from 11.6% to 30.7% class average accuracy).