# Approximate Differentiable Rendering with Algebraic Surfaces

arXiv: https://arxiv.org/abs/2207.10606 · HTML: https://mlsanity.com/paper/2207.10606/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 90 references that are in the map.
- First posted on arXiv 21 Jul 2022.

## Builds on (most cited first, 30 of 30)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction](https://mlsanity.com/paper/2109.00512/) (2021)
- [Modular Primitives for High-Performance Differentiable Rendering](https://mlsanity.com/paper/2011.03277/) (2020)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning](https://mlsanity.com/paper/1904.01786/) (2019)
- [The CMA Evolution Strategy: A Tutorial](https://mlsanity.com/paper/1604.00772/) (2016)
- [Single-View View Synthesis with Multiplane Images](https://mlsanity.com/paper/2004.11364/) (2020)
- [Differentiable Surface Splatting for Point-based Geometry Processing](https://mlsanity.com/paper/1906.04173/) (2019)
- [Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer](https://mlsanity.com/paper/1908.01210/) (2019)
- [Thingi10K: A Dataset of 10,000 3D-Printing Models](https://mlsanity.com/paper/1605.04797/) (2016)
- [MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction](https://mlsanity.com/paper/1703.10580/) (2017)
- [Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction](https://mlsanity.com/paper/1706.07036/) (2017)
- [CubeSLAM: Monocular 3D Object SLAM](https://mlsanity.com/paper/1806.00557/) (2018)
- [Unsupervised Learning of Shape and Pose with Differentiable Point Clouds](https://mlsanity.com/paper/1810.09381/) (2018)
- [Intel RealSense Stereoscopic Depth Cameras](https://mlsanity.com/paper/1705.05548/) (2017)
- [CMA-ES for Hyperparameter Optimization of Deep Neural Networks](https://mlsanity.com/paper/1604.07269/) (2016)
- [End-to-End Learning Local Multi-view Descriptors for 3D Point Clouds](https://mlsanity.com/paper/2003.05855/) (2020)
- [Tangent Space Backpropagation for 3D Transformation Groups](https://mlsanity.com/paper/2103.12032/) (2021)
- [PointGMM: a Neural GMM Network for Point Clouds](https://mlsanity.com/paper/2003.13326/) (2020)
- [Differentiable Surface Rendering via Non-Differentiable Sampling](https://mlsanity.com/paper/2108.04886/) (2021)
- [VoGE: A Differentiable Volume Renderer using Gaussian Ellipsoids for Analysis-by-Synthesis](https://mlsanity.com/paper/2205.15401/) (2022)
- [Mine Tunnel Exploration using Multiple Quadrupedal Robots](https://mlsanity.com/paper/1909.09662/) (2019)
- [GMMLoc: Structure Consistent Visual Localization with Gaussian Mixture Models](https://mlsanity.com/paper/2006.13670/) (2020)
- [MRFMap: Online Probabilistic 3D Mapping using Forward Ray Sensor Models](https://mlsanity.com/paper/2006.03512/) (2020)
- [Direct Fitting of Gaussian Mixture Models](https://mlsanity.com/paper/1904.05537/) (2019)

## Built on by (most cited first, 30 of 44)

- [4D Gaussian Splatting for Real-Time Dynamic Scene Rendering](https://mlsanity.com/paper/2310.08528/) (2023)
- [2D Gaussian Splatting for Geometrically Accurate Radiance Fields](https://mlsanity.com/paper/2403.17888/) (2024)
- [Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction](https://mlsanity.com/paper/2309.13101/) (2023)
- [Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis](https://mlsanity.com/paper/2308.09713/) (2023)
- [Gaussian Splatting SLAM](https://mlsanity.com/paper/2312.06741/) (2023)
- [SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM](https://mlsanity.com/paper/2312.02126/) (2023)
- [SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes](https://mlsanity.com/paper/2312.14937/) (2023)
- [GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation](https://mlsanity.com/paper/2403.14621/) (2024)
- [GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting](https://mlsanity.com/paper/2311.11700/) (2023)
- [Gaussian Opacity Fields: Efficient Adaptive Surface Reconstruction in Unbounded Scenes](https://mlsanity.com/paper/2404.10772/) (2024)
- [GS-IR: 3D Gaussian Splatting for Inverse Rendering](https://mlsanity.com/paper/2311.16473/) (2023)
- [A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets](https://mlsanity.com/paper/2406.12080/) (2024)
- [MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffolds](https://mlsanity.com/paper/2405.17421/) (2024)
- [HUGS: Human Gaussian Splats](https://mlsanity.com/paper/2311.17910/) (2023)
- [3D Gaussian Ray Tracing: Fast Tracing of Particle Scenes](https://mlsanity.com/paper/2407.07090/) (2024)
- [Dynamic Gaussian Marbles for Novel View Synthesis of Casual Monocular Videos](https://mlsanity.com/paper/2406.18717/) (2024)
- [Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splatting](https://mlsanity.com/paper/2402.15870/) (2024)
- [Gaussian Shell Maps for Efficient 3D Human Generation](https://mlsanity.com/paper/2311.17857/) (2023)
- [PhysAvatar: Learning the Physics of Dressed 3D Avatars from Visual Observations](https://mlsanity.com/paper/2404.04421/) (2024)
- [DeformGS: Scene Flow in Highly Deformable Scenes for Deformable Object Manipulation](https://mlsanity.com/paper/2312.00583/) (2023)
- [ClimateNeRF: Extreme Weather Synthesis in Neural Radiance Field](https://mlsanity.com/paper/2211.13226/) (2022)
- [Flexible Techniques for Differentiable Rendering with 3D Gaussians](https://mlsanity.com/paper/2308.14737/) (2023)
- [HybridNeRF: Efficient Neural Rendering via Adaptive Volumetric Surfaces](https://mlsanity.com/paper/2312.03160/) (2023)
- [Gaussian3Diff: 3D Gaussian Diffusion for 3D Full Head Synthesis and Editing](https://mlsanity.com/paper/2312.03763/) (2023)
- [Reconstruct, Inpaint, Test-Time Finetune: Dynamic Novel-view Synthesis from Monocular Videos](https://mlsanity.com/paper/2507.12646/) (2025)
- [GVKF: Gaussian Voxel Kernel Functions for Highly Efficient Surface Reconstruction in Open Scenes](https://mlsanity.com/paper/2411.01853/) (2024)
- [3DGEER: 3D Gaussian Rendering Made Exact and Efficient for Generic Cameras](https://mlsanity.com/paper/2505.24053/) (2025)
- [NeRFs: The Search for the Best 3D Representation](https://mlsanity.com/paper/2308.02751/) (2023)
- [Articulate That Object Part (ATOP): 3D Part Articulation via Text and Motion Personalization](https://mlsanity.com/paper/2502.07278/) (2025)
- [Monocular Dynamic Gaussian Splatting: Fast, Brittle, and Scene Complexity Rules](https://mlsanity.com/paper/2412.04457/) (2024)

## Abstract (arXiv)

Differentiable renderers provide a direct mathematical link between an object's 3D representation and images of that object. In this work, we develop an approximate differentiable renderer for a compact, interpretable representation, which we call Fuzzy Metaballs. Our approximate renderer focuses on rendering shapes via depth maps and silhouettes. It sacrifices fidelity for utility, producing fast runtimes and high-quality gradient information that can be used to solve vision tasks. Compared to mesh-based differentiable renderers, our method has forward passes that are 5x faster and backwards passes that are 30x faster. The depth maps and silhouette images generated by our method are smooth and defined everywhere. In our evaluation of differentiable renderers for pose estimation, we show that our method is the only one comparable to classic techniques. In shape from silhouette, our method performs well using only gradient descent and a per-pixel loss, without any surrogate losses or regularization. These reconstructions work well even on natural video sequences with segmentation artifacts. Project page: https://leonidk.github.io/fuzzy-metaballs