# High Dynamic Range Imaging with Context-aware Transformer

arXiv: https://arxiv.org/abs/2304.04416 · HTML: https://mlsanity.com/paper/2304.04416/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 47 references that are in the map.
- First posted on arXiv 10 Apr 2023.

## Builds on (most cited first, 14 of 14)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Intriguing Properties of Vision Transformers](https://mlsanity.com/paper/2105.10497/) (2021)
- [HDR image reconstruction from a single exposure using deep CNNs](https://mlsanity.com/paper/1710.07480/) (2017)
- [Attention-guided Network for Ghost-free High Dynamic Range Imaging](https://mlsanity.com/paper/1904.10293/) (2019)
- [Deep High Dynamic Range Imaging with Large Foreground Motions](https://mlsanity.com/paper/1711.08937/) (2017)
- [HDR-GAN: HDR Image Reconstruction from Multi-Exposed LDR Images with Large Motions](https://mlsanity.com/paper/2007.01628/) (2020)
- [Ghost-free High Dynamic Range Imaging with Context-aware Transformer](https://mlsanity.com/paper/2208.05114/) (2022)
- [ADNet: Attention-guided Deformable Convolutional Network for High Dynamic Range Imaging](https://mlsanity.com/paper/2105.10697/) (2021)
- [MM-BSN: Self-Supervised Image Denoising for Real-World with Multi-Mask based on Blind-Spot Network](https://mlsanity.com/paper/2304.01598/) (2023)
- [Locally Non-rigid Registration for Mobile HDR Photography](https://mlsanity.com/paper/1504.01441/) (2015)
- [Learning Deformable Kernels for Image and Video Denoising](https://mlsanity.com/paper/1904.06903/) (2019)
- [A Learnable Variational Model for Joint Multimodal MRI Reconstruction and Synthesis](https://mlsanity.com/paper/2204.03804/) (2022)
- [Self-Supervised Image Denoising for Real-World Images with Context-aware Transformer](https://mlsanity.com/paper/2304.01627/) (2023)

## Built on by (most cited first, 3 of 3)

- [Unleashing the Power of Self-Supervised Image Denoising: A Comprehensive Review](https://mlsanity.com/paper/2308.00247/) (2023)
- [A Transfer Learning Framework for Proactive Ramp Metering Performance Assessment](https://mlsanity.com/paper/2308.03542/) (2023)
- [XIMAGENET-12: An Explainable AI Benchmark Dataset for Model Robustness Evaluation](https://mlsanity.com/paper/2310.08182/) (2023)

## Abstract (arXiv)

Avoiding the introduction of ghosts when synthesising LDR images as high dynamic range (HDR) images is a challenging task. Convolutional neural networks (CNNs) are effective for HDR ghost removal in general, but are challenging to deal with the LDR images if there are large movements or oversaturation/undersaturation. Existing dual-branch methods combining CNN and Transformer omit part of the information from non-reference images, while the features extracted by the CNN-based branch are bound to the kernel size with small receptive field, which are detrimental to the deblurring and the recovery of oversaturated/undersaturated regions. In this paper, we propose a novel hierarchical dual Transformer method for ghost-free HDR (HDT-HDR) images generation, which extracts global features and local features simultaneously. First, we use a CNN-based head with spatial attention mechanisms to extract features from all the LDR images. Second, the LDR features are delivered to the Hierarchical Dual Transformer (HDT). In each Dual Transformer (DT), the global features are extracted by the window-based Transformer, while the local details are extracted using the channel attention mechanism with deformable CNNs. Finally, the ghost free HDR image is obtained by dimensional mapping on the HDT output. Abundant experiments demonstrate that our HDT-HDR achieves the state-of-the-art performance among existing HDR ghost removal methods.