# THAT: Token-wise High-frequency Augmentation Transformer for Hyperspectral Pansharpening

arXiv: https://arxiv.org/abs/2508.08183 · HTML: https://mlsanity.com/paper/2508.08183/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 42 references that are in the map.
- First posted on arXiv 11 Aug 2025.

## Builds on (most cited first, 9 of 9)

- [Activating More Pixels in Image Super-Resolution Transformer](https://mlsanity.com/paper/2205.04437/) (2022)
- [Hyperspectral pansharpening: a review](https://mlsanity.com/paper/1504.04531/) (2015)
- [A convex formulation for hyperspectral image superresolution via subspace-based regularization](https://mlsanity.com/paper/1411.4005/) (2014)
- [Deep Gradient Projection Networks for Pan-sharpening](https://mlsanity.com/paper/2103.04584/) (2021)
- [HyperTransformer: A Textural and Spectral Feature Fusion Transformer for Pansharpening](https://mlsanity.com/paper/2203.02503/) (2022)
- [Unsupervised Hyperspectral Pansharpening via Low-rank Diffusion Model](https://mlsanity.com/paper/2305.10925/) (2023)
- [Hyperspectral Pansharpening Based on Improved Deep Image Prior and Residual Reconstruction](https://mlsanity.com/paper/2107.02630/) (2021)
- [Hyperspectral Image Super-Resolution via Deep Prior Regularization with Parameter Estimation](https://mlsanity.com/paper/2009.04237/) (2020)
- [Variational Zero-shot Multispectral Pansharpening](https://mlsanity.com/paper/2407.06633/) (2024)

## Abstract (arXiv)

Transformer-based methods have demonstrated strong potential in hyperspectral pansharpening by modeling long-range dependencies. However, their effectiveness is often limited by redundant token representations and a lack of multi-scale feature modeling. Hyperspectral images exhibit intrinsic spectral priors (e.g., abundance sparsity) and spatial priors (e.g., non-local similarity), which are critical for accurate reconstruction. From a spectral-spatial perspective, Vision Transformers (ViTs) face two major limitations: they struggle to preserve high-frequency components--such as material edges and texture transitions--and suffer from attention dispersion across redundant tokens. These issues stem from the global self-attention mechanism, which tends to dilute high-frequency signals and overlook localized details. To address these challenges, we propose the Token-wise High-frequency Augmentation Transformer (THAT), a novel framework designed to enhance hyperspectral pansharpening through improved high-frequency feature representation and token selection. Specifically, THAT introduces: (1) Pivotal Token Selective Attention (PTSA) to prioritize informative tokens and suppress redundancy; (2) a Multi-level Variance-aware Feed-forward Network (MVFN) to enhance high-frequency detail learning. Experiments on standard benchmarks show that THAT achieves state-of-the-art performance with improved reconstruction quality and efficiency. The source code is available at https://github.com/kailuo93/THAT.