# SkeletonNet: A Topology-Preserving Solution for Learning Mesh Reconstruction of Object Surfaces from RGB Images

arXiv: https://arxiv.org/abs/2008.05742 · HTML: https://mlsanity.com/paper/2008.05742/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Aug 2020.

## Built on by (most cited first, 30 of 30)

- [CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets](https://mlsanity.com/paper/2406.13897/) (2024)
- [Neural Wavelet-domain Diffusion for 3D Shape Generation](https://mlsanity.com/paper/2209.08725/) (2022)
- [SA-ConvONet: Sign-Agnostic Optimization of Convolutional Occupancy Networks](https://mlsanity.com/paper/2105.03582/) (2021)
- [RGBD2: Generative Scene Synthesis via Incremental View Inpainting using RGBD Diffusion Models](https://mlsanity.com/paper/2212.05993/) (2022)
- [Surface Reconstruction from Point Clouds: A Survey and a Benchmark](https://mlsanity.com/paper/2205.02413/) (2022)
- [Neural Template: Topology-aware Reconstruction and Disentangled Generation of 3D Meshes](https://mlsanity.com/paper/2206.04942/) (2022)
- [Generalized Binary Search Network for Highly-Efficient Multi-View Stereo](https://mlsanity.com/paper/2112.02338/) (2021)
- [Learning Parallel Dense Correspondence from Spatio-Temporal Descriptors for Efficient and Robust 4D Reconstruction](https://mlsanity.com/paper/2103.16341/) (2021)
- [Motion2VecSets: 4D Latent Vector Set Diffusion for Non-rigid Shape Reconstruction and Tracking](https://mlsanity.com/paper/2401.06614/) (2024)
- [Neural Shape Deformation Priors](https://mlsanity.com/paper/2210.05616/) (2022)
- [Neural Wavelet-domain Diffusion for 3D Shape Generation, Inversion, and Manipulation](https://mlsanity.com/paper/2302.00190/) (2023)
- [PatchComplete: Learning Multi-Resolution Patch Priors for 3D Shape Completion on Unseen Categories](https://mlsanity.com/paper/2206.04916/) (2022)
- [GSD: View-Guided Gaussian Splatting Diffusion for 3D Reconstruction](https://mlsanity.com/paper/2407.04237/) (2024)
- [T2TD: Text-3D Generation Model based on Prior Knowledge Guidance](https://mlsanity.com/paper/2305.15753/) (2023)
- [GEM3D: GEnerative Medial Abstractions for 3D Shape Synthesis](https://mlsanity.com/paper/2402.16994/) (2024)
- [Coverage Axis++: Efficient Inner Point Selection for 3D Shape Skeletonization](https://mlsanity.com/paper/2401.12946/) (2024)
- [What's the Situation with Intelligent Mesh Generation: A Survey and Perspectives](https://mlsanity.com/paper/2211.06009/) (2022)
- [RGB2Point: 3D Point Cloud Generation from Single RGB Images](https://mlsanity.com/paper/2407.14979/) (2024)
- [3D Shape Completion on Unseen Categories:A Weakly-supervised Approach](https://mlsanity.com/paper/2401.10578/) (2024)
- [PVSeRF: Joint Pixel-, Voxel- and Surface-Aligned Radiance Field for Single-Image Novel View Synthesis](https://mlsanity.com/paper/2202.04879/) (2022)
- [WuYun: Exploring hierarchical skeleton-guided melody generation using knowledge-enhanced deep learning](https://mlsanity.com/paper/2301.04488/) (2023)
- [PHRIT: Parametric Hand Representation with Implicit Template](https://mlsanity.com/paper/2309.14916/) (2023)
- [Interactive Sketching of Mannequin Poses](https://mlsanity.com/paper/2212.07098/) (2022)
- [Visual Enhanced 3D Point Cloud Reconstruction from A Single Image](https://mlsanity.com/paper/2108.07685/) (2021)
- [Visibility-Aware Pixelwise View Selection for Multi-View Stereo Matching](https://mlsanity.com/paper/2302.07182/) (2023)
- [Neural Vector Fields: Generalizing Distance Vector Fields by Codebooks and Zero-Curl Regularization](https://mlsanity.com/paper/2309.01512/) (2023)
- [Leveraging Automatic CAD Annotations for Supervised Learning in 3D Scene Understanding](https://mlsanity.com/paper/2504.13580/) (2025)
- [Edge Preserving Implicit Surface Representation of Point Clouds](https://mlsanity.com/paper/2301.04860/) (2023)
- [Relational Anatomical Supervision for Accurate 3D Multi-Chamber Cardiac Mesh Reconstruction](https://mlsanity.com/paper/2503.07874/) (2025)
- [Shape2.5D: A Dataset of Texture-less Surfaces for Depth and Normals Estimation](https://mlsanity.com/paper/2406.15831/) (2024)

## Abstract (arXiv)

This paper focuses on the challenging task of learning 3D object surface reconstructions from RGB images. Existingmethods achieve varying degrees of success by using different surface representations. However, they all have their own drawbacks,and cannot properly reconstruct the surface shapes of complex topologies, arguably due to a lack of constraints on the topologicalstructures in their learning frameworks. To this end, we propose to learn and use the topology-preserved, skeletal shape representationto assist the downstream task of object surface reconstruction from RGB images. Technically, we propose the novelSkeletonNetdesign that learns a volumetric representation of a skeleton via a bridged learning of a skeletal point set, where we use paralleldecoders each responsible for the learning of points on 1D skeletal curves and 2D skeletal sheets, as well as an efficient module ofglobally guided subvolume synthesis for a refined, high-resolution skeletal volume; we present a differentiablePoint2Voxellayer tomake SkeletonNet end-to-end and trainable. With the learned skeletal volumes, we propose two models, the Skeleton-Based GraphConvolutional Neural Network (SkeGCNN) and the Skeleton-Regularized Deep Implicit Surface Network (SkeDISN), which respectivelybuild upon and improve over the existing frameworks of explicit mesh deformation and implicit field learning for the downstream surfacereconstruction task. We conduct thorough experiments that verify the efficacy of our proposed SkeletonNet. SkeGCNN and SkeDISNoutperform existing methods as well, and they have their own merits when measured by different metrics. Additional results ingeneralized task settings further demonstrate the usefulness of our proposed methods. We have made both our implementation codeand the ShapeNet-Skeleton dataset publicly available at ble at https://github.com/tangjiapeng/SkeletonNet.