# Depth-to-RGB: Repurposing a Frozen Depth Estimator for Geometry-Guided Compositing

arXiv: https://arxiv.org/abs/2610.09125 · HTML: https://mlsanity.com/paper/2610.09125/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 78 references that are in the map.
- First posted on arXiv 6 Oct 2026.

## Builds on (most cited first, 6 of 6)

- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [Qwen-Image Technical Report](https://mlsanity.com/paper/2508.02324/) (2025)
- [Modality Forcing for Scalable Spatial Generation](https://mlsanity.com/paper/2606.13676/) (2026)
- [InsertFuse: A Unified Framework for Multi-Category Reference-Guided Image Insertion](https://mlsanity.com/paper/2608.06490/) (2026)

## Abstract (arXiv)

Reference-based object compositing inserts or replaces an object using a background image, a reference image, and a 2D compositing mask. These inputs guide appearance and placement but leave the completed scene's geometry implicit, which can distort object structure or alter the surroundings. Our Depth-to-RGB (D2R) framework predicts composite depth for a scene not yet observed in the RGB inputs. It learns reference-conditioned corrections to a frozen depth estimator using encoder features of paired completed scenes as targets. The unchanged decoder maps the corrected representation to the intended scene's depth, which a separately trained renderer holds fixed during RGB synthesis. Under matched architecture and training, encoder-feature supervision reduces OOD Stage-1 AbsRel by 31.4% relative to decoded-depth supervision. We also introduce AnyInsertion++ with paired in-distribution and category-disjoint splits to evaluate generalization beyond compositing training categories. The complete D2R system leads 12 open-source and 3 closed-source baselines in estimator-derived geometry and photometric quality on both paired splits. On category-disjoint data, D2R reduces AbsRel by 43.7% and improves PSNR by 2.4 dB over the matched RGB baseline. Across three unpaired benchmarks, D2R leads both identity metrics and reduces mean CLIP reference cosine distance by 55% relative to the strongest baseline. Project page: https://shjo-april.github.io/Depth2RGB/