Differentiable Rendering: A Survey
Not on the map. Why?
Abstract (arXiv)
Deep neural networks (DNNs) have shown remarkable performance improvements on vision-related tasks such as object detection or image segmentation. Despite their success, they generally lack the understanding of 3D objects which form the image, as it is not always possible to collect 3D information about the scene or to easily annotate it. Differentiable rendering is a novel field which allows the gradients of 3D objects to be calculated and propagated through images. It also reduces the requirement of 3D data collection and annotation, while enabling higher success rate in various applications. This paper reviews existing literature and discusses the current state of differentiable rendering, its applications and open research problems.
Key facts. Cited by 123 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Jun 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 22; 2022: 34; 2023: 22; 2024: 21; 2025: 9; 2026: 15.
123 papers.
Sort:
By area:
- 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
- 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 370
- 2022NeILF: Neural Incident Light Field for Physically-based Material Estimationcited by 105
- 2021Unsupervised Discovery of Object Radiance Fieldscited by 100
- 2021Gradients are Not All You Needcited by 82
- 2021Learning Signed Distance Field for Multi-view Surface Reconstructioncited by 76
- 2021Unified Shape and SVBRDF Recovery using Differentiable Monte Carlo Renderingcited by 71
- 2024Advances in 3D Generation: A Surveycited by 62
- 2021Paint Transformer: Feed Forward Neural Painting with Stroke Predictioncited by 61
- 2022A Survey of Non-Rigid 3D Registrationcited by 59
- 2022Improving the Transferability of Targeted Adversarial Examples through Object-Based Diverse Inputcited by 49
- 2021Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Drivingcited by 45
38 more
- 2023RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head Avatarscited by 45
- 2022CLIP-Actor: Text-Driven Recommendation and Stylization for Animating Human Meshescited by 41
- 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Modelscited by 41
- 2021Modern Evolution Strategies for Creativity: Fitting Concrete Images and Abstract Conceptscited by 40
- 2022Occlusion-Aware Self-Supervised Monocular 6D Object Pose Estimationcited by 34
- 2021Recursive-NeRF: An Efficient and Dynamically Growing NeRFcited by 32
- 2022DL4SciVis: A State-of-the-Art Survey on Deep Learning for Scientific Visualizationcited by 32
- 2023BeyondPixels: A Comprehensive Review of the Evolution of Neural Radiance Fieldscited by 27
- 2022Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistencycited by 26
- 2023CVTHead: One-shot Controllable Head Avatar with Vertex-feature Transformercited by 24
- 20223DFaceShop: Explicitly Controllable 3D-Aware Portrait Generationcited by 22
- 2022SAM-RL: Sensing-Aware Model-Based Reinforcement Learning via Differentiable Physics-Based Simulation and Renderingcited by 22
- 2022High-Res Facial Appearance Capture from Polarized Smartphone Imagescited by 21
- 2024GausSurf: Geometry-Guided 3D Gaussian Splatting for Surface Reconstructioncited by 19
- 2023MoSAR: Monocular Semi-Supervised Model for Avatar Reconstruction using Differentiable Shadingcited by 18
- 2023Re-ReND: Real-time Rendering of NeRFs across Devicescited by 17
- 2025AirExo-2: Scaling up Generalizable Robotic Imitation Learning with Low-Cost Exoskeletonscited by 17
- 2023PAPR: Proximity Attention Point Renderingcited by 16
- 2023NeRRF: 3D Reconstruction and View Synthesis for Transparent and Specular Objects with Neural Refractive-Reflective Fieldscited by 16
- 2025Reconstructing 4D Spatial Intelligence: A Surveycited by 13
- 2022GenDR: A Generalized Differentiable Renderercited by 12
- 2024Fed3DGS: Scalable 3D Gaussian Splatting with Federated Learningcited by 12
- 2023Style2Fab: Functionality-Aware Segmentation for Fabricating Personalized 3D Models with Generative AIcited by 11
- 2021Shadow Art Revisited: A Differentiable Rendering Based Approachcited by 10
- 2022Plateau-reduced Differentiable Path Tracingcited by 10
- 2023NeRF Revisited: Fixing Quadrature Instability in Volume Renderingcited by 10
- 2025Materialist: Physically Based Editing Using Single-Image Inverse Renderingcited by 10
- 2021ADOP: Approximate Differentiable One-Pixel Point Renderingcited by 9
- 2023Image-based Pose Estimation and Shape Reconstruction for Robot Manipulators and Soft, Continuum Robots via Differentiable Renderingcited by 9
- 2023Multi-view 3D Object Reconstruction and Uncertainty Modelling with Neural Shape Priorcited by 9
- 2022Cluttered Food Grasping with Adaptive Fingers and Synthetic-Data Trained Object Detectioncited by 8
- 2022Leveraging Single-View Images for Unsupervised 3D Point Cloud Completioncited by 8
- 2021An Effective Loss Function for Generating 3D Models from Single 2D Image without Renderingcited by 7
- 2023SSR-2D: Semantic 3D Scene Reconstruction from 2D Imagescited by 7
- 2023DiViNeT: 3D Reconstruction from Disparate Views via Neural Template Regularizationcited by 7
- 2024Source-Free and Image-Only Unsupervised Domain Adaptation for Category Level Object Pose Estimationcited by 7
- 2021Physics-based Differentiable Depth Sensor Simulationcited by 6
- 2022ROAD: Learning an Implicit Recursive Octree Auto-Decoder to Efficiently Encode 3D Shapescited by 6
50 most cited of 123 All 123 (JSON)
- 2026Benchmarking Optimizers to Solve Inverse Problems with Differentiable Physics Simulators12 Sep 2026
- 20263D Point Splatting for mmWave Radar Novel View Synthesis10 Sep 2026
- 2026Inverse Rendering for Modeling with Line Primitives1 Sep 2026
- 2026FaceSnap: Real-Time Personalized Lightstage Facial Performance Capture31 Aug 2026
- 2026mmIR: Frequency-Space Inverse Rendering for 3D Millimeter-Wave Radar ADC Synthesis28 Aug 2026
- 2026ReconSplat: Generalizable 3D Scene Reconstruction Beyond Observed Views28 Aug 2026
- 2026Mosaic: A Benchmark Suite for Differentiable Physics Solvers26 Jun 2026
- 2026A Cookbook of 3D Vision: Data, Learning Paradigms, and Application2 Jun 2026
- 2026Implicit Neural Representations: A Signal Processing Perspective16 Apr 2026
- 2026DiffSoup: Direct Differentiable Rasterization of Triangle Soup for Extreme Radiance Field Simplification28 Mar 2026
- 2026Physically Accurate Differentiable Inverse Rendering for Radio Frequency Digital Twin5 Mar 2026
- 2026Simulation-Ready Cluttered Scene Estimation via Physics-aware Joint Shape and Pose Optimization23 Feb 2026
38 more
- 2026Gaussian Mesh Renderer for Lightweight Differentiable Rendering16 Feb 2026
- 2026Deep Sketch-Based 3D Modeling: A Survey22 Jan 2026
- 2026GAT-NeRF: Geometry-Aware-Transformer Enhanced Neural Radiance Fields for High-Fidelity 4D Facial Avatars21 Jan 2026
- 2025Neural Image Abstraction Using Long Smoothing B-Splines7 Nov 2025
- 2025GaussianGAN: Real-Time Photorealistic controllable Human Avatars1 Sep 2025
- 2025Disentangled Geometry and Appearance for Efficient Multi-View Surface Reconstruction and Rendering24 Aug 2025
- 2025Reconstructing 4D Spatial Intelligence: A Survey28 Jul 2025
- 2025Reinforced Embodied Active Defense: Exploiting Adaptive Interaction for Robust Visual Perception in Adversarial 3D Environments24 Jul 2025
- 2025Advances in Radiance Field for Dynamic Scene: From Neural Field to Gaussian Field15 May 2025
- 2025AirExo-2: Scaling up Generalizable Robotic Imitation Learning with Low-Cost Exoskeletons5 Mar 2025
- 2025Light Transport-aware Diffusion Posterior Sampling for Single-View Reconstruction of 3D Volumes9 Jan 2025
- 2025Materialist: Physically Based Editing Using Single-Image Inverse Rendering7 Jan 2025
- 2024SEREP: Semantic Facial Expression Representation for Robust In-the-Wild Capture and Retargeting18 Dec 2024
- 2024GausSurf: Geometry-Guided 3D Gaussian Splatting for Surface Reconstruction29 Nov 2024
- 2024Imperceptible Adversarial Examples in the Physical World25 Nov 2024
- 2024RenderBender: A Survey on Adversarial Attacks Using Differentiable Rendering14 Nov 2024
- 2024Triplet: Triangle Patchlet for Mesh-Based Inverse Rendering and Scene Parameters Approximation16 Oct 2024
- 2024From Goal-Conditioned to Language-Conditioned Agents via Vision-Language Models24 Sep 2024
- 2024Towards Real-Time Generation of Delay-Compensated Video Feeds for Outdoor Mobile Robot Teleoperation16 Sep 2024
- 2024Rethinking Directional Parameterization in Neural Implicit Surface Reconstruction11 Sep 2024
- 2024Progressive Radiance Distillation for Inverse Rendering with Gaussian Splatting14 Aug 2024
- 2024Differentiable Voxelization and Mesh Morphing15 Jul 2024
- 2024Optimizing Automatic Differentiation with Deep Reinforcement Learning7 Jun 2024
- 2024ReFiNe: Recursive Field Networks for Cross-modal Multi-scene Representation6 Jun 2024
- 2024PILA: Physics-Informed Low Rank Augmentation for Interpretable Earth Observation29 May 2024
- 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models16 May 2024
- 2024DiffGen: Robot Demonstration Generation via Differentiable Physics Simulation, Differentiable Rendering, and Vision-Language Model12 May 2024
- 2024Embodied Active Defense: Leveraging Recurrent Feedback to Counter Adversarial Patches31 Mar 2024
- 2024Fed3DGS: Scalable 3D Gaussian Splatting with Federated Learning18 Mar 2024
- 2024Bayesian Differentiable Physics for Cloth Digitalization27 Feb 2024
- 2024Advances in 3D Generation: A Survey31 Jan 2024
- 2024Source-Free and Image-Only Unsupervised Domain Adaptation for Category Level Object Pose Estimation19 Jan 2024
- 2024High-Quality Mesh Blendshape Generation from Face Videos via Neural Inverse Rendering16 Jan 2024
- 2023MoSAR: Monocular Semi-Supervised Model for Avatar Reconstruction using Differentiable Shading20 Dec 2023
- 2023Differentiable Radio Frequency Ray Tracing for Millimeter-Wave Sensing22 Nov 2023
- 2023LocaliseBot: Multi-view 3D object localisation with differentiable rendering for robot grasping14 Nov 2023
- 2023CVTHead: One-shot Controllable Head Avatar with Vertex-feature Transformer11 Nov 2023
- 2023NeRF Revisited: Fixing Quadrature Instability in Volume Rendering31 Oct 2023
50 newest of 123 All 123 (JSON)
- 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Modelscited by 41
- 2024DiffGen: Robot Demonstration Generation via Differentiable Physics Simulation, Differentiable Rendering, and Vision-Language Modelcited by 2
- 2024From Goal-Conditioned to Language-Conditioned Agents via Vision-Language Modelscited by 0
- 2024From Goal-Conditioned to Language-Conditioned Agents via Vision-Language Models24 Sep 2024
- 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models16 May 2024
- 2024DiffGen: Robot Demonstration Generation via Differentiable Physics Simulation, Differentiable Rendering, and Vision-Language Model12 May 2024
- 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
- 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 370
- 2022NeILF: Neural Incident Light Field for Physically-based Material Estimationcited by 105
- 2021Unsupervised Discovery of Object Radiance Fieldscited by 100
- 2021Gradients are Not All You Needcited by 82
- 2021Learning Signed Distance Field for Multi-view Surface Reconstructioncited by 76
- 2021Unified Shape and SVBRDF Recovery using Differentiable Monte Carlo Renderingcited by 71
- 2024Advances in 3D Generation: A Surveycited by 62
- 2021Paint Transformer: Feed Forward Neural Painting with Stroke Predictioncited by 61
- 2022A Survey of Non-Rigid 3D Registrationcited by 59
- 2022Improving the Transferability of Targeted Adversarial Examples through Object-Based Diverse Inputcited by 49
- 2021Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Drivingcited by 45
12 most cited of 118 All 118 (JSON)
- 2026Benchmarking Optimizers to Solve Inverse Problems with Differentiable Physics Simulators12 Sep 2026
- 20263D Point Splatting for mmWave Radar Novel View Synthesis10 Sep 2026
- 2026Inverse Rendering for Modeling with Line Primitives1 Sep 2026
- 2026FaceSnap: Real-Time Personalized Lightstage Facial Performance Capture31 Aug 2026
- 2026mmIR: Frequency-Space Inverse Rendering for 3D Millimeter-Wave Radar ADC Synthesis28 Aug 2026
- 2026ReconSplat: Generalizable 3D Scene Reconstruction Beyond Observed Views28 Aug 2026
- 2026Mosaic: A Benchmark Suite for Differentiable Physics Solvers26 Jun 2026
- 2026A Cookbook of 3D Vision: Data, Learning Paradigms, and Application2 Jun 2026
- 2026Implicit Neural Representations: A Signal Processing Perspective16 Apr 2026
- 2026DiffSoup: Direct Differentiable Rasterization of Triangle Soup for Extreme Radiance Field Simplification28 Mar 2026
- 2026Physically Accurate Differentiable Inverse Rendering for Radio Frequency Digital Twin5 Mar 2026
- 2026Simulation-Ready Cluttered Scene Estimation via Physics-aware Joint Shape and Pose Optimization23 Feb 2026
12 newest of 118 All 118 (JSON)