# PASTA: Towards Flexible and Efficient HDR Imaging Via Progressively Aggregated Spatio-Temporal Alignment

arXiv: https://arxiv.org/abs/2403.10376 · HTML: https://mlsanity.com/paper/2403.10376/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 68 references that are in the map.
- First posted on arXiv 15 Mar 2024.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Generative Diffusion Prior for Unified Image Restoration and Enhancement](https://mlsanity.com/paper/2304.01247/) (2023)
- [Attention-guided Network for Ghost-free High Dynamic Range Imaging](https://mlsanity.com/paper/1904.10293/) (2019)
- [Deep High Dynamic Range Imaging with Large Foreground Motions](https://mlsanity.com/paper/1711.08937/) (2017)
- [HDR-GAN: HDR Image Reconstruction from Multi-Exposed LDR Images with Large Motions](https://mlsanity.com/paper/2007.01628/) (2020)
- [Ghost-free High Dynamic Range Imaging with Context-aware Transformer](https://mlsanity.com/paper/2208.05114/) (2022)
- [ADNet: Attention-guided Deformable Convolutional Network for High Dynamic Range Imaging](https://mlsanity.com/paper/2105.10697/) (2021)
- [WaveFill: A Wavelet-based Generation Network for Image Inpainting](https://mlsanity.com/paper/2107.11027/) (2021)
- [Single Image Depth Prediction with Wavelet Decomposition](https://mlsanity.com/paper/2106.02022/) (2021)
- [Alignment-free HDR Deghosting with Semantics Consistent Transformer](https://mlsanity.com/paper/2305.18135/) (2023)
- [Attention-Guided Progressive Neural Texture Fusion for High Dynamic Range Image Restoration](https://mlsanity.com/paper/2107.06211/) (2021)
- [Locally Non-rigid Registration for Mobile HDR Photography](https://mlsanity.com/paper/1504.01441/) (2015)
- [FlexHDR: Modelling Alignment and Exposure Uncertainties for Flexible HDR Imaging](https://mlsanity.com/paper/2201.02625/) (2022)
- [A Unified HDR Imaging Method with Pixel and Patch Level](https://mlsanity.com/paper/2304.06943/) (2023)
- [Progressive and Selective Fusion Network for High Dynamic Range Imaging](https://mlsanity.com/paper/2108.08585/) (2021)
- [SMAE: Few-shot Learning for HDR Deghosting with Saturation-Aware Masked Autoencoders](https://mlsanity.com/paper/2304.06914/) (2023)
- [Wavelet-Based Network For High Dynamic Range Imaging](https://mlsanity.com/paper/2108.01434/) (2021)
- [Self-Gated Memory Recurrent Network for Efficient Scalable HDR Deghosting](https://mlsanity.com/paper/2112.13050/) (2021)
- [DRHDR: A Dual branch Residual Network for Multi-Bracket High Dynamic Range Imaging](https://mlsanity.com/paper/2206.04124/) (2022)
- [CEN-HDR: Computationally Efficient neural Network for real-time High Dynamic Range imaging](https://mlsanity.com/paper/2302.05213/) (2023)

## Built on by (most cited first, 3 of 3)

- [Training Transformer Models by Wavelet Losses Improves Quantitative and Visual Performance in Single Image Super-Resolution](https://mlsanity.com/paper/2404.11273/) (2024)
- [DRNet: All-in-One Image Restoration via Prior-Guided Dynamic Reparameterization](https://mlsanity.com/paper/2605.08627/) (2026)
- [There and Back Again: A Flexible-Frame Transformer for Multi-Exposure Fusion](https://mlsanity.com/paper/2606.27905/) (2026)

## Abstract (arXiv)

Leveraging Transformer attention has led to great advancements in HDR deghosting. However, the intricate nature of self-attention introduces practical challenges, as existing state-of-the-art methods often demand high-end GPUs or exhibit slow inference speeds, especially for high-resolution images like 2K. Striking an optimal balance between performance and latency remains a critical concern. In response, this work presents PASTA, a novel Progressively Aggregated Spatio-Temporal Alignment framework for HDR deghosting. Our approach achieves effectiveness and efficiency by harnessing hierarchical representation during feature distanglement. Through the utilization of diverse granularities within the hierarchical structure, our method substantially boosts computational speed and optimizes the HDR imaging workflow. In addition, we explore within-scale feature modeling with local and global attention, gradually merging and refining them in a coarse-to-fine fashion. Experimental results showcase PASTA's superiority over current SOTA methods in both visual quality and performance metrics, accompanied by a substantial 3-fold (x3) increase in inference speed.