# SilNet : Single- and Multi-View Reconstruction by Learning from Silhouettes

arXiv: https://arxiv.org/abs/1711.07888 · HTML: https://mlsanity.com/paper/1711.07888/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Nov 2017.

## Built on by (most cited first, 25 of 25)

- [ABO: Dataset and Benchmarks for Real-World 3D Object Understanding](https://mlsanity.com/paper/2110.06199/) (2021)
- [Shelf-Supervised Mesh Prediction in the Wild](https://mlsanity.com/paper/2102.06195/) (2021)
- [FiG-NeRF: Figure-Ground Neural Radiance Fields for 3D Object Category Modelling](https://mlsanity.com/paper/2104.08418/) (2021)
- [Learning Generative Models of Textured 3D Meshes from Real-World Images](https://mlsanity.com/paper/2103.15627/) (2021)
- [SAIL-VOS 3D: A Synthetic Dataset and Baselines for Object Detection and 3D Mesh Reconstruction from Video Data](https://mlsanity.com/paper/2105.08612/) (2021)
- [Shape and Material Capture at Home](https://mlsanity.com/paper/2104.06397/) (2021)
- [Pre-train, Self-train, Distill: A simple recipe for Supersizing 3D Reconstruction](https://mlsanity.com/paper/2204.03642/) (2022)
- [MVPSNet: Fast Generalizable Multi-view Photometric Stereo](https://mlsanity.com/paper/2305.11167/) (2023)
- [StrobeNet: Category-Level Multiview Reconstruction of Articulated Objects](https://mlsanity.com/paper/2105.08016/) (2021)
- [Fast Light-Weight Near-Field Photometric Stereo](https://mlsanity.com/paper/2203.16515/) (2022)
- [Structural Causal 3D Reconstruction](https://mlsanity.com/paper/2207.10156/) (2022)
- [Deep Learning Methods for Calibrated Photometric Stereo and Beyond](https://mlsanity.com/paper/2212.08414/) (2022)
- [Neural Architecture Search for Efficient Uncalibrated Deep Photometric Stereo](https://mlsanity.com/paper/2110.05621/) (2021)
- [Multi-Category Mesh Reconstruction From Image Collections](https://mlsanity.com/paper/2110.11256/) (2021)
- [RASP: Revisiting 3D Anamorphic Art for Shadow-Guided Packing of Irregular Objects](https://mlsanity.com/paper/2504.02465/) (2025)
- [RMAFF-PSN: A Residual Multi-Scale Attention Feature Fusion Photometric Stereo Network](https://mlsanity.com/paper/2404.07766/) (2024)
- [Image Gradient-Aided Photometric Stereo Network](https://mlsanity.com/paper/2412.11650/) (2024)
- [Deep Uncalibrated Photometric Stereo via Inter-Intra Image Feature Fusion](https://mlsanity.com/paper/2208.03440/) (2022)
- [NeuralMPS: Non-Lambertian Multispectral Photometric Stereo via Spectral Reflectance Decomposition](https://mlsanity.com/paper/2211.15311/) (2022)
- [Event Fusion Photometric Stereo Network](https://mlsanity.com/paper/2303.00308/) (2023)
- [Two-stage Synthetic Supervising and Multi-view Consistency Self-supervising based Animal 3D Reconstruction by Single Image](https://mlsanity.com/paper/2311.13199/) (2023)
- [Refine3DNet: Scaling Precision in 3D Object Reconstruction from Multi-View RGB Images using Attention](https://mlsanity.com/paper/2412.00731/) (2024)
- [MSF-Net: Multi-Stage Feature Extraction and Fusion for Robust Photometric Stereo](https://mlsanity.com/paper/2510.25221/) (2025)
- [Event-based Photometric Stereo via Rotating Illumination and Per-Pixel Learning](https://mlsanity.com/paper/2603.10748/) (2026)
- [PIE-PS: Photometric Stereo from Physical Irradiance Event Streams](https://mlsanity.com/paper/2610.08188/) (2026)

## Abstract (arXiv)

The objective of this paper is 3D shape understanding from single and multiple images. To this end, we introduce a new deep-learning architecture and loss function, SilNet, that can handle multiple views in an order-agnostic manner. The architecture is fully convolutional, and for training we use a proxy task of silhouette prediction, rather than directly learning a mapping from 2D images to 3D shape as has been the target in most recent work. We demonstrate that with the SilNet architecture there is generalisation over the number of views -- for example, SilNet trained on 2 views can be used with 3 or 4 views at test-time; and performance improves with more views. We introduce two new synthetics datasets: a blobby object dataset useful for pre-training, and a challenging and realistic sculpture dataset; and demonstrate on these datasets that SilNet has indeed learnt 3D shape. Finally, we show that SilNet exceeds the state of the art on the ShapeNet benchmark dataset, and use SilNet to generate novel views of the sculpture dataset.