# Locally Orderless Images for Optimization in Differentiable Rendering

arXiv: https://arxiv.org/abs/2503.21931 · HTML: https://mlsanity.com/paper/2503.21931/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 49 references that are in the map.
- First posted on arXiv 27 Mar 2025.

## Builds on (most cited first, 15 of 15)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Computational Optimal Transport](https://mlsanity.com/paper/1803.00567/) (2018)
- [Modular Primitives for High-Performance Differentiable Rendering](https://mlsanity.com/paper/2011.03277/) (2020)
- [Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning](https://mlsanity.com/paper/1904.01786/) (2019)
- [Do Differentiable Simulators Give Better Policy Gradients?](https://mlsanity.com/paper/2202.00817/) (2022)
- [Gradients are Not All You Need](https://mlsanity.com/paper/2111.05803/) (2021)
- [Factorized Inverse Path Tracing for Efficient and Accurate Material-Lighting Estimation](https://mlsanity.com/paper/2304.05669/) (2023)
- [Plateau-reduced Differentiable Path Tracing](https://mlsanity.com/paper/2211.17263/) (2022)
- [A Simple Approach to Differentiable Rendering of SDFs](https://mlsanity.com/paper/2405.08733/) (2024)
- [A Theory of Topological Derivatives for Inverse Rendering of Geometry](https://mlsanity.com/paper/2308.09865/) (2023)
- [Zero Grads: Learning Local Surrogate Losses for Non-Differentiable Graphics](https://mlsanity.com/paper/2308.05739/) (2023)
- [Transforming a Non-Differentiable Rasterizer into a Differentiable One with Stochastic Gradient Estimation](https://mlsanity.com/paper/2404.09758/) (2024)

## Built on by (most cited first, 1 of 1)

- [Feature-Guided Diffusion for Non-Differentiable Inverse Rendering](https://mlsanity.com/paper/2607.17411/) (2026)

## Abstract (arXiv)

Problems in differentiable rendering often involve optimizing scene parameters that cause motion in image space. The gradients for such parameters tend to be sparse, leading to poor convergence. While existing methods address this sparsity through proxy gradients such as topological derivatives or lagrangian derivatives, they make simplifying assumptions about rendering. Multi-resolution image pyramids offer an alternative approach but prove unreliable in practice. We introduce a method that uses locally orderless images, where each pixel maps to a histogram of intensities that preserves local variations in appearance. Using an inverse rendering objective that minimizes histogram distance, our method extends support for sparsely defined image gradients and recovers optimal parameters. We validate our method on various inverse problems using both synthetic and real data.