# Deformable kernel networks for guided depth map upsampling

arXiv: https://arxiv.org/abs/1903.11286 · HTML: https://mlsanity.com/paper/1903.11286/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2019.

## Built on by (most cited first, 7 of 7)

- [DCCF: Deep Comprehensible Color Filter Learning Framework for High-Resolution Image Harmonization](https://mlsanity.com/paper/2207.04788/) (2022)
- [BridgeNet: A Joint Learning Network of Depth Map Super-Resolution and Monocular Depth Estimation](https://mlsanity.com/paper/2107.12541/) (2021)
- [Learning Scene Structure Guidance via Cross-Task Knowledge Transfer for Single Depth Super-Resolution](https://mlsanity.com/paper/2103.12955/) (2021)
- [Symmetric Uncertainty-Aware Feature Transmission for Depth Super-Resolution](https://mlsanity.com/paper/2306.00386/) (2023)
- [Revisiting Deformable Convolution for Depth Completion](https://mlsanity.com/paper/2308.01905/) (2023)
- [IGAF: Incremental Guided Attention Fusion for Depth Super-Resolution](https://mlsanity.com/paper/2501.01723/) (2025)
- [Guided Depth Map Super-Resolution via Multi-Scale Fusion U-shaped Mamba Network](https://mlsanity.com/paper/2508.00248/) (2025)

## Abstract (arXiv)

We address the problem of upsampling a low-resolution (LR) depth map using a registered high-resolution (HR) color image of the same scene. Previous methods based on convolutional neural networks (CNNs) combine nonlinear activations of spatially-invariant kernels to estimate structural details from LR depth and HR color images, and regress upsampling results directly from the networks. In this paper, we revisit the weighted averaging process that has been widely used to transfer structural details from hand-crafted visual features to LR depth maps. We instead learn explicitly sparse and spatially-variant kernels for this task. To this end, we propose a CNN architecture and its efficient implementation, called the deformable kernel network (DKN), that outputs sparse sets of neighbors and the corresponding weights adaptively for each pixel. We also propose a fast version of DKN (FDKN) that runs about 17 times faster (0.01 seconds for a HR image of size 640 x 480). Experimental results on standard benchmarks demonstrate the effectiveness of our approach. In particular, we show that the weighted averaging process with 3 x 3 kernels (i.e., aggregating 9 samples sparsely chosen) outperforms the state of the art by a significant margin.