# DRWR: A Differentiable Renderer without Rendering for Unsupervised 3D Structure Learning from Silhouette Images

arXiv: https://arxiv.org/abs/2007.06127 · HTML: https://mlsanity.com/paper/2007.06127/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jul 2020.

## Built on by (most cited first, 30 of 35)

- [SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformer](https://mlsanity.com/paper/2108.04444/) (2021)
- [Differentiable Point-Based Radiance Fields for Efficient View Synthesis](https://mlsanity.com/paper/2205.14330/) (2022)
- [Reconstructing Surfaces for Sparse Point Clouds with On-Surface Priors](https://mlsanity.com/paper/2204.10603/) (2022)
- [PMP-Net++: Point Cloud Completion by Transformer-Enhanced Multi-step Point Moving Paths](https://mlsanity.com/paper/2202.09507/) (2022)
- [Surface Reconstruction from Point Clouds by Learning Predictive Context Priors](https://mlsanity.com/paper/2204.11015/) (2022)
- [Cycle4Completion: Unpaired Point Cloud Completion using Cycle Transformation with Missing Region Coding](https://mlsanity.com/paper/2103.07838/) (2021)
- [SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fields](https://mlsanity.com/paper/2212.02501/) (2022)
- [3D Shape Reconstruction from 2D Images with Disentangled Attribute Flow](https://mlsanity.com/paper/2203.15190/) (2022)
- [Learning Deep Implicit Functions for 3D Shapes with Dynamic Code Clouds](https://mlsanity.com/paper/2203.14048/) (2022)
- [Towards Better Gradient Consistency for Neural Signed Distance Functions via Level Set Alignment](https://mlsanity.com/paper/2305.11601/) (2023)
- [Snowflake Point Deconvolution for Point Cloud Completion and Generation with Skip-Transformer](https://mlsanity.com/paper/2202.09367/) (2022)
- [Fast Learning Radiance Fields by Shooting Much Fewer Rays](https://mlsanity.com/paper/2208.06821/) (2022)
- [NeuroFluid: Fluid Dynamics Grounding with Particle-Driven Neural Radiance Fields](https://mlsanity.com/paper/2203.01762/) (2022)
- [Learning a More Continuous Zero Level Set in Unsigned Distance Fields through Level Set Projection](https://mlsanity.com/paper/2308.11441/) (2023)
- [Unsupervised Inference of Signed Distance Functions from Single Sparse Point Clouds without Learning Priors](https://mlsanity.com/paper/2303.14505/) (2023)
- [Learning Signed Distance Functions from Noisy 3D Point Clouds via Noise to Noise Mapping](https://mlsanity.com/paper/2306.01405/) (2023)
- [Unsupervised Learning of Fine Structure Generation for 3D Point Clouds by 2D Projection Matching](https://mlsanity.com/paper/2108.03746/) (2021)
- [Latent Partition Implicit with Surface Codes for 3D Representation](https://mlsanity.com/paper/2207.08631/) (2022)
- [Learning 3D Object Shape and Layout without 3D Supervision](https://mlsanity.com/paper/2206.07028/) (2022)
- [Fast Learning of Signed Distance Functions from Noisy Point Clouds via Noise to Noise Mapping](https://mlsanity.com/paper/2407.14225/) (2024)
- [Parts2Words: Learning Joint Embedding of Point Clouds and Texts by Bidirectional Matching between Parts and Words](https://mlsanity.com/paper/2107.01872/) (2021)
- [GridPull: Towards Scalability in Learning Implicit Representations from 3D Point Clouds](https://mlsanity.com/paper/2308.13175/) (2023)
- [MultiPull: Detailing Signed Distance Functions by Pulling Multi-Level Queries at Multi-Step](https://mlsanity.com/paper/2411.01208/) (2024)
- [Single-view 3D Mesh Reconstruction for Seen and Unseen Categories](https://mlsanity.com/paper/2208.02676/) (2022)
- [ADOP: Approximate Differentiable One-Pixel Point Rendering](https://mlsanity.com/paper/2110.06635/) (2021)
- [Leveraging Single-View Images for Unsupervised 3D Point Cloud Completion](https://mlsanity.com/paper/2212.00564/) (2022)
- [Hierarchical View Predictor: Unsupervised 3D Global Feature Learning through Hierarchical Prediction among Unordered Views](https://mlsanity.com/paper/2108.03743/) (2021)
- [Inferring Neural Signed Distance Functions by Overfitting on Single Noisy Point Clouds through Finetuning Data-Driven based Priors](https://mlsanity.com/paper/2410.19680/) (2024)
- [Flow-based GAN for 3D Point Cloud Generation from a Single Image](https://mlsanity.com/paper/2210.04072/) (2022)
- [Learning Local Pattern Modularization for Point Cloud Reconstruction from Unseen Classes](https://mlsanity.com/paper/2408.14279/) (2024)

## Abstract (arXiv)

Differentiable renderers have been used successfully for unsupervised 3D structure learning from 2D images because they can bridge the gap between 3D and 2D. To optimize 3D shape parameters, current renderers rely on pixel-wise losses between rendered images of 3D reconstructions and ground truth images from corresponding viewpoints. Hence they require interpolation of the recovered 3D structure at each pixel, visibility handling, and optionally evaluating a shading model. In contrast, here we propose a Differentiable Renderer Without Rendering (DRWR) that omits these steps. DRWR only relies on a simple but effective loss that evaluates how well the projections of reconstructed 3D point clouds cover the ground truth object silhouette. Specifically, DRWR employs a smooth silhouette loss to pull the projection of each individual 3D point inside the object silhouette, and a structure-aware repulsion loss to push each pair of projections that fall inside the silhouette far away from each other. Although we omit surface interpolation, visibility handling, and shading, our results demonstrate that DRWR achieves state-of-the-art accuracies under widely used benchmarks, outperforming previous methods both qualitatively and quantitatively. In addition, our training times are significantly lower due to the simplicity of DRWR.