# Recurrent Structure Attention Guidance for Depth Super-Resolution

arXiv: https://arxiv.org/abs/2301.13419 · HTML: https://mlsanity.com/paper/2301.13419/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 46 references that are in the map.
- First posted on arXiv 31 Jan 2023.

## Builds on (most cited first, 19 of 19)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans](https://mlsanity.com/paper/1812.07003/) (2018)
- [Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution](https://mlsanity.com/paper/1904.05049/) (2019)
- [Pixel-Adaptive Convolutional Neural Networks](https://mlsanity.com/paper/1904.05373/) (2019)
- [Closed-loop Matters: Dual Regression Networks for Single Image Super-Resolution](https://mlsanity.com/paper/2003.07018/) (2020)
- [The Fast Bilateral Solver](https://mlsanity.com/paper/1511.03296/) (2015)
- [Deep Convolutional Neural Network for Multi-modal Image Restoration and Fusion](https://mlsanity.com/paper/1910.04066/) (2019)
- [Deformable Kernel Networks for Joint Image Filtering](https://mlsanity.com/paper/1910.08373/) (2019)
- [Regularizing Nighttime Weirdness: Efficient Self-supervised Monocular Depth Estimation in the Dark](https://mlsanity.com/paper/2108.03830/) (2021)
- [Joint Image Filtering with Deep Convolutional Networks](https://mlsanity.com/paper/1710.04200/) (2017)
- [Channel Attention based Iterative Residual Learning for Depth Map Super-Resolution](https://mlsanity.com/paper/2006.01469/) (2020)
- [BridgeNet: A Joint Learning Network of Depth Map Super-Resolution and Monocular Depth Estimation](https://mlsanity.com/paper/2107.12541/) (2021)
- [Guided Image-to-Image Translation with Bi-Directional Feature Transformation](https://mlsanity.com/paper/1910.11328/) (2019)
- [Learning Scene Structure Guidance via Cross-Task Knowledge Transfer for Single Depth Super-Resolution](https://mlsanity.com/paper/2103.12955/) (2021)
- [High-resolution Depth Maps Imaging via Attention-based Hierarchical Multi-modal Fusion](https://mlsanity.com/paper/2104.01530/) (2021)
- [Joint convolutional neural pyramid for depth map super-resolution](https://mlsanity.com/paper/1801.00968/) (2018)

## Built on by (most cited first, 16 of 16)

- [SGNet: Structure Guided Network via Gradient-Frequency Awareness for Depth Map Super-Resolution](https://mlsanity.com/paper/2312.05799/) (2023)
- [DORNet: A Degradation Oriented and Regularized Network for Blind Depth Super-Resolution](https://mlsanity.com/paper/2410.11666/) (2024)
- [RGB Guided ToF Imaging System: A Survey of Deep Learning-based Methods](https://mlsanity.com/paper/2405.10357/) (2024)
- [Decoupling Fine Detail and Global Geometry for Compressed Depth Map Super-Resolution](https://mlsanity.com/paper/2411.03239/) (2024)
- [Depth Super-Resolution from Explicit and Implicit High-Frequency Features](https://mlsanity.com/paper/2303.09307/) (2023)
- [Scene Prior Filtering for Depth Super-Resolution](https://mlsanity.com/paper/2402.13876/) (2024)
- [SVDC: Consistent Direct Time-of-Flight Video Depth Completion with Frequency Selective Fusion](https://mlsanity.com/paper/2503.01257/) (2025)
- [SpatioTemporal Difference Network for Video Depth Super-Resolution](https://mlsanity.com/paper/2508.01259/) (2025)
- [MoCoLSK: Modality Conditioned High-Resolution Downscaling for Land Surface Temperature](https://mlsanity.com/paper/2409.19835/) (2024)
- [NAIMA: Semantics Aware RGB Guided Depth Super-Resolution](https://mlsanity.com/paper/2604.04407/) (2026)
- [Guided Depth Map Super-Resolution via Multi-Scale Fusion U-shaped Mamba Network](https://mlsanity.com/paper/2508.00248/) (2025)
- [2D Representation for Unguided Single-View 3D Super-Resolution in Real-Time](https://mlsanity.com/paper/2511.08224/) (2025)
- [Robust Depth Super-Resolution via Adaptive Diffusion Sampling](https://mlsanity.com/paper/2602.09510/) (2026)
- [Interactive State Space Model with Cross-Modal Local Scanning for Depth Super-Resolution](https://mlsanity.com/paper/2605.11934/) (2026)
- [WAVE: Reversing the Guidance Hierarchy for Coarse-to-Fine Guided Depth Super-Resolution](https://mlsanity.com/paper/2608.25302/) (2026)
- [Depth-to-Image Synthesis-Driven Generative Unguided Depth Completion](https://mlsanity.com/paper/2609.06007/) (2026)

## Abstract (arXiv)

Image guidance is an effective strategy for depth super-resolution. Generally, most existing methods employ hand-crafted operators to decompose the high-frequency (HF) and low-frequency (LF) ingredients from low-resolution depth maps and guide the HF ingredients by directly concatenating them with image features. However, the hand-designed operators usually cause inferior HF maps (e.g., distorted or structurally missing) due to the diverse appearance of complex depth maps. Moreover, the direct concatenation often results in weak guidance because not all image features have a positive effect on the HF maps. In this paper, we develop a recurrent structure attention guided (RSAG) framework, consisting of two important parts. First, we introduce a deep contrastive network with multi-scale filters for adaptive frequency-domain separation, which adopts contrastive networks from large filters to small ones to calculate the pixel contrasts for adaptive high-quality HF predictions. Second, instead of the coarse concatenation guidance, we propose a recurrent structure attention block, which iteratively utilizes the latest depth estimation and the image features to jointly select clear patterns and boundaries, aiming at providing refined guidance for accurate depth recovery. In addition, we fuse the features of HF maps to enhance the edge structures in the decomposed LF maps. Extensive experiments show that our approach obtains superior performance compared with state-of-the-art depth super-resolution methods.