# Integrating Spatial and Frequency Information for Under-Display Camera Image Restoration

arXiv: https://arxiv.org/abs/2501.18517 · HTML: https://mlsanity.com/paper/2501.18517/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 50 references that are in the map.
- First posted on arXiv 30 Jan 2025.

## Builds on (most cited first, 29 of 29)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [Rethinking Coarse-to-Fine Approach in Single Image Deblurring](https://mlsanity.com/paper/2108.05054/) (2021)
- [Multi-Scale Progressive Fusion Network for Single Image Deraining](https://mlsanity.com/paper/2003.10985/) (2020)
- [Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset](https://mlsanity.com/paper/1904.01538/) (2019)
- [Efficient Frequency Domain-based Transformers for High-Quality Image Deblurring](https://mlsanity.com/paper/2211.12250/) (2022)
- [Defocus Deblurring Using Dual-Pixel Data](https://mlsanity.com/paper/2005.00305/) (2020)
- [Dense Extreme Inception Network: Towards a Robust CNN Model for Edge Detection](https://mlsanity.com/paper/1909.01955/) (2019)
- [FourLLIE: Boosting Low-Light Image Enhancement by Fourier Frequency Information](https://mlsanity.com/paper/2308.03033/) (2023)
- [Flare7K: A Phenomenological Nighttime Flare Removal Dataset](https://mlsanity.com/paper/2210.06570/) (2022)
- [Image Restoration for Under-Display Camera](https://mlsanity.com/paper/2003.04857/) (2020)
- [Towards Real Scene Super-Resolution with Raw Images](https://mlsanity.com/paper/1905.12156/) (2019)
- [Removing Diffraction Image Artifacts in Under-Display Camera via Dynamic Skip Connection Network](https://mlsanity.com/paper/2104.09556/) (2021)
- [Nighttime Smartphone Reflective Flare Removal Using Optical Center Symmetry Prior](https://mlsanity.com/paper/2303.15046/) (2023)
- [Improving Lens Flare Removal with General Purpose Pipeline and Multiple Light Sources Recovery](https://mlsanity.com/paper/2308.16460/) (2023)
- [Generating Aligned Pseudo-Supervision from Non-Aligned Data for Image Restoration in Under-Display Camera](https://mlsanity.com/paper/2304.06019/) (2023)
- [UDC 2020 Challenge on Image Restoration of Under-Display Camera: Methods and Results](https://mlsanity.com/paper/2008.07742/) (2020)
- [Under-Display Camera Image Restoration with Scattering Effect](https://mlsanity.com/paper/2308.04163/) (2023)
- [Transform Domain Pyramidal Dilated Convolution Networks For Restoration of Under Display Camera Images](https://mlsanity.com/paper/2009.09393/) (2020)
- [UDC-UNet: Under-Display Camera Image Restoration via U-Shape Dynamic Network](https://mlsanity.com/paper/2209.01809/) (2022)
- [Spatial-frequency Dual-Domain Feature Fusion Network for Low-Light Remote Sensing Image Enhancement](https://mlsanity.com/paper/2404.17400/) (2024)

## Built on by (most cited first, 1 of 1)

- [MERLIN: Building Low-SNR Robust Multimodal LLMs for Electromagnetic Signals](https://mlsanity.com/paper/2603.08174/) (2026)

## Abstract (arXiv)

Under-Display Camera (UDC) houses a digital camera lens under a display panel. However, UDC introduces complex degradations such as noise, blur, decrease in transmittance, and flare. Despite the remarkable progress, previous research on UDC mainly focuses on eliminating diffraction in the spatial domain and rarely explores its potential in the frequency domain. It is essential to consider both the spatial and frequency domains effectively. For example, degradations, such as noise and blur, can be addressed by local information (e.g., CNN kernels in the spatial domain). At the same time, tackling flares may require leveraging global information (e.g., the frequency domain). In this paper, we revisit the UDC degradations in the Fourier space and figure out intrinsic frequency priors that imply the presence of the flares. Based on this observation, we propose a novel multi-level DNN architecture called SFIM. It efficiently restores UDC-distorted images by integrating local and global (the collective contribution of all points in the image) information. The architecture exploits CNNs to capture local information and FFT-based models to capture global information. SFIM comprises a spatial domain block (SDB), a Frequency Domain Block (FDB), and an Attention-based Multi-level Integration Block (AMIB). Specifically, SDB focuses more on detailed textures such as noise and blur, FDB emphasizes irregular texture loss in extensive areas such as flare, and AMIB enables effective cross-domain interaction. SFIM's superior performance over state-of-the-art approaches is demonstrated through rigorous quantitative and qualitative assessments across three UDC benchmarks.