# Dressi: A Hardware-Agnostic Differentiable Renderer with Reactive Shader Packing and Soft Rasterization

arXiv: https://arxiv.org/abs/2204.01386 · HTML: https://mlsanity.com/paper/2204.01386/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 117 references that are in the map.
- First posted on arXiv 4 Apr 2022.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Modular Primitives for High-Performance Differentiable Rendering](https://mlsanity.com/paper/2011.03277/) (2020)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning](https://mlsanity.com/paper/1904.01786/) (2019)
- [3D Morphable Face Models -- Past, Present and Future](https://mlsanity.com/paper/1909.01815/) (2019)
- [DiffTaichi: Differentiable Programming for Physical Simulation](https://mlsanity.com/paper/1910.00935/) (2019)
- [Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer](https://mlsanity.com/paper/1908.01210/) (2019)
- [Learning to Predict Indoor Illumination from a Single Image](https://mlsanity.com/paper/1704.00090/) (2017)
- [GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction](https://mlsanity.com/paper/1902.05978/) (2019)
- [Differentiable Rendering: A Survey](https://mlsanity.com/paper/2006.12057/) (2020)
- [Inverse Path Tracing for Joint Material and Lighting Estimation](https://mlsanity.com/paper/1903.07145/) (2019)
- [High-Fidelity 3D Digital Human Head Creation from RGB-D Selfies](https://mlsanity.com/paper/2010.05562/) (2020)
- [The Deep Learning Compiler: A Comprehensive Survey](https://mlsanity.com/paper/2002.03794/) (2020)
- [A Versatile Scene Model with Differentiable Visibility Applied to Generative Pose Estimation](https://mlsanity.com/paper/1602.03725/) (2016)
- [Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction](https://mlsanity.com/paper/2105.07474/) (2021)
- [TensorFlow.js: Machine Learning for the Web and Beyond](https://mlsanity.com/paper/1901.05350/) (2019)
- [Instead of Rewriting Foreign Code for Machine Learning, Automatically Synthesize Fast Gradients](https://mlsanity.com/paper/2010.01709/) (2020)
- [Appearance-Driven Automatic 3D Model Simplification](https://mlsanity.com/paper/2104.03989/) (2021)
- [Opt: A Domain Specific Language for Non-linear Least Squares Optimization in Graphics and Imaging](https://mlsanity.com/paper/1604.06525/) (2016)

## Built on by (most cited first, 2 of 2)

- [Dr.Hair: Reconstructing Scalp-Connected Hair Strands without Pre-training via Differentiable Rendering of Line Segments](https://mlsanity.com/paper/2403.17496/) (2024)
- [Differentiable Shadow Mapping for Efficient Inverse Graphics](https://mlsanity.com/paper/2308.10896/) (2023)

## Abstract (arXiv)

Differentiable rendering (DR) enables various computer graphics and computer vision applications through gradient-based optimization with derivatives of the rendering equation. Most rasterization-based approaches are built on general-purpose automatic differentiation (AD) libraries and DR-specific modules handcrafted using CUDA. Such a system design mixes DR algorithm implementation and algorithm building blocks, resulting in hardware dependency and limited performance. In this paper, we present a practical hardware-agnostic differentiable renderer called Dressi, which is based on a new full AD design. The DR algorithms of Dressi are fully written in our Vulkan-based AD for DR, Dressi-AD, which supports all primitive operations for DR. Dressi-AD and our inverse UV technique inside it bring hardware independence and acceleration by graphics hardware. Stage packing, our runtime optimization technique, can adapt hardware constraints and efficiently execute complex computational graphs of DR with reactive cache considering the render pass hierarchy of Vulkan. HardSoftRas, our novel rendering process, is designed for inverse rendering with a graphics pipeline. Under the limited functionalities of the graphics pipeline, HardSoftRas can propagate the gradients of pixels from the screen space to far-range triangle attributes. Our experiments and applications demonstrate that Dressi establishes hardware independence, high-quality and robust optimization with fast speed, and photorealistic rendering.