# Differentiable Direct Volume Rendering

arXiv: https://arxiv.org/abs/2107.12672 · HTML: https://mlsanity.com/paper/2107.12672/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jul 2021.

## Built on by (most cited first, 15 of 15)

- [Compressed 3D Gaussian Splatting for Accelerated Novel View Synthesis](https://mlsanity.com/paper/2401.02436/) (2023)
- [Fast Neural Representations for Direct Volume Rendering](https://mlsanity.com/paper/2112.01579/) (2021)
- [Differentiable Visual Computing for Inverse Problems and Machine Learning](https://mlsanity.com/paper/2312.04574/) (2023)
- [GSCache: Real-Time Radiance Caching for Volume Path Tracing using 3D Gaussian Splatting](https://mlsanity.com/paper/2507.19718/) (2025)
- [Sliding Gaussian ball adaptive growth (SlingBAG): point cloud-based iterative algorithm for large-scale 3D photoacoustic imaging](https://mlsanity.com/paper/2407.11781/) (2024)
- [Uncertainty-Informed Volume Visualization using Implicit Neural Representation](https://mlsanity.com/paper/2408.06018/) (2024)
- [Monocular Depth Decomposition of Semi-Transparent Volume Renderings](https://mlsanity.com/paper/2206.13282/) (2022)
- [Lightweight Gradient-Aware Upscaling of 3D Gaussian Splatting Images](https://mlsanity.com/paper/2503.14171/) (2025)
- [Three-dimensional visualization of X-ray micro-CT with large-scale datasets: Efficiency and accuracy for real-time interaction](https://mlsanity.com/paper/2601.15098/) (2026)
- [Hybrid Latents: Geometry-Appearance-Aware Surfel Splatting](https://mlsanity.com/paper/2604.14928/) (2026)
- [Differentiable Electron Microscopy Simulation: Methods and Applications for Visualization](https://mlsanity.com/paper/2205.04464/) (2022)
- [Light Transport-aware Diffusion Posterior Sampling for Single-View Reconstruction of 3D Volumes](https://mlsanity.com/paper/2501.05226/) (2025)
- [GPAIR: Gaussian-Kernel-Based Ultrafast 3D Photoacoustic Iterative Reconstruction](https://mlsanity.com/paper/2602.03893/) (2026)
- [NeuVolEx: Implicit Neural Features for Volume Exploration](https://mlsanity.com/paper/2604.11172/) (2026)
- [FactorSplat: Appearance-Controllable Gaussian Proxies for Medical Volume Rendering](https://mlsanity.com/paper/2610.02382/) (2026)

## Abstract (arXiv)

We present a differentiable volume rendering solution that provides differentiability of all continuous parameters of the volume rendering process. This differentiable renderer is used to steer the parameters towards a setting with an optimal solution of a problem-specific objective function. We have tailored the approach to volume rendering by enforcing a constant memory footprint via analytic inversion of the blending functions. This makes it independent of the number of sampling steps through the volume and facilitates the consideration of small-scale changes. The approach forms the basis for automatic optimizations regarding external parameters of the rendering process and the volumetric density field itself. We demonstrate its use for automatic viewpoint selection using differentiable entropy as objective, and for optimizing a transfer function from rendered images of a given volume. Optimization of per-voxel densities is addressed in two different ways: First, we mimic inverse tomography and optimize a 3D density field from images using an absorption model. This simplification enables comparisons with algebraic reconstruction techniques and state-of-the-art differentiable path tracers. Second, we introduce a novel approach for tomographic reconstruction from images using an emission-absorption model with post-shading via an arbitrary transfer function.