# Depth Super-Resolution from Explicit and Implicit High-Frequency Features

arXiv: https://arxiv.org/abs/2303.09307 · HTML: https://mlsanity.com/paper/2303.09307/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 82 references that are in the map.
- First posted on arXiv 16 Mar 2023.

## Builds on (most cited first, 30 of 41)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [A Survey on Visual Transformer](https://mlsanity.com/paper/2012.12556/) (2020)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation](https://mlsanity.com/paper/1903.01945/) (2019)
- [Dynamic Convolution: Attention over Convolution Kernels](https://mlsanity.com/paper/1912.03458/) (2019)
- [Cascaded Pyramid Network for Multi-Person Pose Estimation](https://mlsanity.com/paper/1711.07319/) (2017)
- [Non-Local Spatial Propagation Network for Depth Completion](https://mlsanity.com/paper/2007.10042/) (2020)
- [FcaNet: Frequency Channel Attention Networks](https://mlsanity.com/paper/2012.11879/) (2020)
- [Pixel-Adaptive Convolutional Neural Networks](https://mlsanity.com/paper/1904.05373/) (2019)
- [3D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Prior](https://mlsanity.com/paper/2003.14052/) (2020)
- [MPViT: Multi-Path Vision Transformer for Dense Prediction](https://mlsanity.com/paper/2112.11010/) (2021)
- [Efficient Image Super-Resolution Using Pixel Attention](https://mlsanity.com/paper/2010.01073/) (2020)
- [Deep Convolutional Neural Network for Multi-modal Image Restoration and Fusion](https://mlsanity.com/paper/1910.04066/) (2019)
- [Discrete Cosine Transform Network for Guided Depth Map Super-Resolution](https://mlsanity.com/paper/2104.06977/) (2021)
- [Towards Fast and Accurate Real-World Depth Super-Resolution: Benchmark Dataset and Baseline](https://mlsanity.com/paper/2104.06174/) (2021)
- [Delving Deeper into Anti-aliasing in ConvNets](https://mlsanity.com/paper/2008.09604/) (2020)
- [Deformable Kernel Networks for Joint Image Filtering](https://mlsanity.com/paper/1910.08373/) (2019)
- [Joint Image Filtering with Deep Convolutional Networks](https://mlsanity.com/paper/1710.04200/) (2017)
- [Guided Depth Super-Resolution by Deep Anisotropic Diffusion](https://mlsanity.com/paper/2211.11592/) (2022)
- [DPANet: Depth Potentiality-Aware Gated Attention Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2003.08608/) (2020)

## Built on by (most cited first, 5 of 5)

- [SGNet: Structure Guided Network via Gradient-Frequency Awareness for Depth Map Super-Resolution](https://mlsanity.com/paper/2312.05799/) (2023)
- [RGB Guided ToF Imaging System: A Survey of Deep Learning-based Methods](https://mlsanity.com/paper/2405.10357/) (2024)
- [Cross-spectral Gated-RGB Stereo Depth Estimation](https://mlsanity.com/paper/2405.12759/) (2024)
- [Learnable Fractional Reaction-Diffusion Dynamics for Under-Display ToF Imaging and Beyond](https://mlsanity.com/paper/2511.01704/) (2025)
- [Enhanced Encoder-Decoder Architecture for Accurate Monocular Depth Estimation](https://mlsanity.com/paper/2410.11610/) (2024)

## Abstract (arXiv)

We propose a novel multi-stage depth super-resolution network, which progressively reconstructs high-resolution depth maps from explicit and implicit high-frequency features. The former are extracted by an efficient transformer processing both local and global contexts, while the latter are obtained by projecting color images into the frequency domain. Both are combined together with depth features by means of a fusion strategy within a multi-stage and multi-scale framework. Experiments on the main benchmarks, such as NYUv2, Middlebury, DIML and RGBDD, show that our approach outperforms existing methods by a large margin (~20% on NYUv2 and DIML against the contemporary work DADA, with 16x upsampling), establishing a new state-of-the-art in the guided depth super-resolution task.