# Modular Degradation Simulation and Restoration for Under-Display Camera

arXiv: https://arxiv.org/abs/2209.11455 · HTML: https://mlsanity.com/paper/2209.11455/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 67 references that are in the map.
- First posted on arXiv 23 Sep 2022.

## Builds on (most cited first, 30 of 45)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [MetaFormer Is Actually What You Need for Vision](https://mlsanity.com/paper/2111.11418/) (2021)
- [Learning to See in the Dark](https://mlsanity.com/paper/1805.01934/) (2018)
- [DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks](https://mlsanity.com/paper/1711.07064/) (2017)
- [DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better](https://mlsanity.com/paper/1908.03826/) (2019)
- [Toward Convolutional Blind Denoising of Real Photographs](https://mlsanity.com/paper/1807.04686/) (2018)
- [Geometric GAN](https://mlsanity.com/paper/1705.02894/) (2017)
- [Learning a Convolutional Neural Network for Non-uniform Motion Blur Removal](https://mlsanity.com/paper/1503.00593/) (2015)
- [A Physics-based Noise Formation Model for Extreme Low-light Raw Denoising](https://mlsanity.com/paper/2003.12751/) (2020)
- [A U-Net Based Discriminator for Generative Adversarial Networks](https://mlsanity.com/paper/2002.12655/) (2020)
- [Dual Adversarial Network: Toward Real-world Noise Removal and Noise Generation](https://mlsanity.com/paper/2007.05946/) (2020)
- [Semantic Image Synthesis via Adversarial Learning](https://mlsanity.com/paper/1707.06873/) (2017)
- [A Neural Approach to Blind Motion Deblurring](https://mlsanity.com/paper/1603.04771/) (2016)

## Built on by (most cited first, 7 of 7)

- [Under-Display Camera Image Restoration with Scattering Effect](https://mlsanity.com/paper/2308.04163/) (2023)
- [Learning a Practical SDR-to-HDRTV Up-conversion using New Dataset and Degradation Models](https://mlsanity.com/paper/2303.13031/) (2023)
- [LRDif: Diffusion Models for Under-Display Camera Emotion Recognition](https://mlsanity.com/paper/2402.00250/) (2024)
- [Blind Face Restoration for Under-Display Camera via Dictionary Guided Transformer](https://mlsanity.com/paper/2308.10196/) (2023)
- [Deep Video Restoration for Under-Display Camera](https://mlsanity.com/paper/2309.04752/) (2023)
- [Adversarial Purification and Fine-tuning for Robust UDC Image Restoration](https://mlsanity.com/paper/2402.13629/) (2024)
- [UCMNet: Uncertainty-Aware Context Memory Network for Under-Display Camera Image Restoration](https://mlsanity.com/paper/2604.00381/) (2026)

## Abstract (arXiv)

Under-display camera (UDC) provides an elegant solution for full-screen smartphones. However, UDC captured images suffer from severe degradation since sensors lie under the display. Although this issue can be tackled by image restoration networks, these networks require large-scale image pairs for training. To this end, we propose a modular network dubbed MPGNet trained using the generative adversarial network (GAN) framework for simulating UDC imaging. Specifically, we note that the UDC imaging degradation process contains brightness attenuation, blurring, and noise corruption. Thus we model each degradation with a characteristic-related modular network, and all modular networks are cascaded to form the generator. Together with a pixel-wise discriminator and supervised loss, we can train the generator to simulate the UDC imaging degradation process. Furthermore, we present a Transformer-style network named DWFormer for UDC image restoration. For practical purposes, we use depth-wise convolution instead of the multi-head self-attention to aggregate local spatial information. Moreover, we propose a novel channel attention module to aggregate global information, which is critical for brightness recovery. We conduct evaluations on the UDC benchmark, and our method surpasses the previous state-of-the-art models by 1.23 dB on the P-OLED track and 0.71 dB on the T-OLED track, respectively.