# Exploiting Frequency Correlation for Hyperspectral Image Reconstruction

arXiv: https://arxiv.org/abs/2406.00683 · HTML: https://mlsanity.com/paper/2406.00683/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 80 references that are in the map.
- First posted on arXiv 2 Jun 2024.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks](https://mlsanity.com/paper/1905.13545/) (2019)
- [Understanding Robustness of Transformers for Image Classification](https://mlsanity.com/paper/2103.14586/) (2021)
- [Focal Frequency Loss for Image Reconstruction and Synthesis](https://mlsanity.com/paper/2012.12821/) (2020)
- [Training behavior of deep neural network in frequency domain](https://mlsanity.com/paper/1807.01251/) (2018)
- [Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction](https://mlsanity.com/paper/2111.07910/) (2021)
- [Accurate Image Restoration with Attention Retractable Transformer](https://mlsanity.com/paper/2210.01427/) (2022)
- [Unsupervised Real-world Image Super Resolution via Domain-distance Aware Training](https://mlsanity.com/paper/2004.01178/) (2020)
- [CiaoSR: Continuous Implicit Attention-in-Attention Network for Arbitrary-Scale Image Super-Resolution](https://mlsanity.com/paper/2212.04362/) (2022)
- [Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction](https://mlsanity.com/paper/2203.04845/) (2022)
- [HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging](https://mlsanity.com/paper/2203.02149/) (2022)
- [Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging](https://mlsanity.com/paper/2205.10102/) (2022)
- [Deep Gaussian Scale Mixture Prior for Spectral Compressive Imaging](https://mlsanity.com/paper/2103.07152/) (2021)
- [GAP-net for Snapshot Compressive Imaging](https://mlsanity.com/paper/2012.08364/) (2020)
- [Diversity in Machine Learning](https://mlsanity.com/paper/1807.01477/) (2018)
- [HyperTransformer: A Textural and Spectral Feature Fusion Transformer for Pansharpening](https://mlsanity.com/paper/2203.02503/) (2022)
- [FPGA: Fast Patch-Free Global Learning Framework for Fully End-to-End Hyperspectral Image Classification](https://mlsanity.com/paper/2011.05670/) (2020)
- [HerosNet: Hyperspectral Explicable Reconstruction and Optimal Sampling Deep Network for Snapshot Compressive Imaging](https://mlsanity.com/paper/2112.06238/) (2021)
- [Basic Binary Convolution Unit for Binarized Image Restoration Network](https://mlsanity.com/paper/2210.00405/) (2022)
- [Residual Degradation Learning Unfolding Framework with Mixing Priors across Spectral and Spatial for Compressive Spectral Imaging](https://mlsanity.com/paper/2211.06891/) (2022)

## Built on by (most cited first, 2 of 2)

- [Complementary Advantages: Exploiting Cross-Field Frequency Correlation for NIR-Assisted Image Denoising](https://mlsanity.com/paper/2412.16645/) (2024)
- [DCT-Mamba3D: Spectral Decorrelation and Spatial-Spectral Feature Extraction for Hyperspectral Image Classification](https://mlsanity.com/paper/2502.01986/) (2025)

## Abstract (arXiv)

Deep priors have emerged as potent methods in hyperspectral image (HSI) reconstruction. While most methods emphasize space-domain learning using image space priors like non-local similarity, frequency-domain learning using image frequency priors remains neglected, limiting the reconstruction capability of networks. In this paper, we first propose a Hyperspectral Frequency Correlation (HFC) prior rooted in in-depth statistical frequency analyses of existent HSI datasets. Leveraging the HFC prior, we subsequently establish the frequency domain learning composed of a Spectral-wise self-Attention of Frequency (SAF) and a Spectral-spatial Interaction of Frequency (SIF) targeting low-frequency and high-frequency components, respectively. The outputs of SAF and SIF are adaptively merged by a learnable gating filter, thus achieving a thorough exploitation of image frequency priors. Integrating the frequency domain learning and the existing space domain learning, we finally develop the Correlation-driven Mixing Domains Transformer (CMDT) for HSI reconstruction. Extensive experiments highlight that our method surpasses various state-of-the-art (SOTA) methods in reconstruction quality and computational efficiency.