# RoSI: Recovering 3D Shape Interiors from Few Articulation Images

arXiv: https://arxiv.org/abs/2304.06342 · HTML: https://mlsanity.com/paper/2304.06342/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 49 references that are in the map.
- First posted on arXiv 13 Apr 2023.

## Builds on (most cited first, 30 of 35)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [SAPIEN: A SimulAted Part-based Interactive ENvironment](https://mlsanity.com/paper/2003.08515/) (2020)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [PCN: Point Completion Network](https://mlsanity.com/paper/1808.00671/) (2018)
- [ABO: Dataset and Benchmarks for Real-World 3D Object Understanding](https://mlsanity.com/paper/2110.06199/) (2021)
- [Deep Closest Point: Learning Representations for Point Cloud Registration](https://mlsanity.com/paper/1905.03304/) (2019)
- [3D-FUTURE: 3D Furniture shape with TextURE](https://mlsanity.com/paper/2009.09633/) (2020)
- [PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers](https://mlsanity.com/paper/2108.08839/) (2021)
- [Category-Level Articulated Object Pose Estimation](https://mlsanity.com/paper/1912.11913/) (2019)
- [PF-Net: Point Fractal Network for 3D Point Cloud Completion](https://mlsanity.com/paper/2003.00410/) (2020)
- [A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representation](https://mlsanity.com/paper/2104.07645/) (2021)
- [ShapeFormer: Transformer-based Shape Completion via Sparse Representation](https://mlsanity.com/paper/2201.10326/) (2022)
- [PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Image](https://mlsanity.com/paper/1812.04072/) (2018)
- [Variational Relational Point Completion Network](https://mlsanity.com/paper/2104.10154/) (2021)
- [CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Clouds](https://mlsanity.com/paper/2104.03437/) (2021)
- [SeedFormer: Patch Seeds based Point Cloud Completion with Upsample Transformer](https://mlsanity.com/paper/2207.10315/) (2022)
- [PlaneNet: Piece-wise Planar Reconstruction from a Single RGB Image](https://mlsanity.com/paper/1804.06278/) (2018)
- [Single-Image Piece-wise Planar 3D Reconstruction via Associative Embedding](https://mlsanity.com/paper/1902.09777/) (2019)
- [OPD: Single-view 3D Openable Part Detection](https://mlsanity.com/paper/2203.16421/) (2022)
- [Self-supervised Neural Articulated Shape and Appearance Models](https://mlsanity.com/paper/2205.08525/) (2022)
- [Planar Surface Reconstruction from Sparse Views](https://mlsanity.com/paper/2103.14644/) (2021)
- [Pose from Shape: Deep Pose Estimation for Arbitrary 3D Objects](https://mlsanity.com/paper/1906.05105/) (2019)
- [Towards Real-World Category-level Articulation Pose Estimation](https://mlsanity.com/paper/2105.03260/) (2021)
- [Snowflake Point Deconvolution for Point Cloud Completion and Generation with Skip-Transformer](https://mlsanity.com/paper/2202.09367/) (2022)
- [Understanding 3D Object Articulation in Internet Videos](https://mlsanity.com/paper/2203.16531/) (2022)
- [PlaneTR: Structure-Guided Transformers for 3D Plane Recovery](https://mlsanity.com/paper/2107.13108/) (2021)

## Built on by (most cited first, 6 of 6)

- [S2O: Static to Openable Enhancement for Articulated 3D Objects](https://mlsanity.com/paper/2409.18896/) (2024)
- [FreeArt3D: Training-Free Articulated Object Generation using 3D Diffusion](https://mlsanity.com/paper/2510.25765/) (2025)
- [LARM: A Large Articulated-Object Reconstruction Model](https://mlsanity.com/paper/2511.11563/) (2025)
- [MonoMobility: Zero-Shot 3D Mobility Analysis from Monocular Videos](https://mlsanity.com/paper/2505.11868/) (2025)
- [ArtPro: Self-Supervised Articulated Object Reconstruction with Adaptive Integration of Mobility Proposals](https://mlsanity.com/paper/2602.22666/) (2026)
- [DailyArt: Discovering Articulation from Single Static Images via Latent Dynamics](https://mlsanity.com/paper/2604.07758/) (2026)

## Abstract (arXiv)

The dominant majority of 3D models that appear in gaming, VR/AR, and those we use to train geometric deep learning algorithms are incomplete, since they are modeled as surface meshes and missing their interior structures. We present a learning framework to recover the shape interiors (RoSI) of existing 3D models with only their exteriors from multi-view and multi-articulation images. Given a set of RGB images that capture a target 3D object in different articulated poses, possibly from only few views, our method infers the interior planes that are observable in the input images. Our neural architecture is trained in a category-agnostic manner and it consists of a motion-aware multi-view analysis phase including pose, depth, and motion estimations, followed by interior plane detection in images and 3D space, and finally multi-view plane fusion. In addition, our method also predicts part articulations and is able to realize and even extrapolate the captured motions on the target 3D object. We evaluate our method by quantitative and qualitative comparisons to baselines and alternative solutions, as well as testing on untrained object categories and real image inputs to assess its generalization capabilities.