FOCUS: Fused Observation of Channels for Unveiling Spectra
Not on the map. Why?
Abstract (arXiv)
Hyperspectral imaging (HSI) captures hundreds of narrow, contiguous wavelength bands, making it a powerful tool in biology, agriculture, and environmental monitoring. However, interpreting Vision Transformers (ViTs) in this setting remains largely unexplored due to two key challenges: (1) existing saliency methods struggle to capture meaningful spectral cues, often collapsing attention onto the class token, and (2) full-spectrum ViTs are computationally prohibitive for interpretability, given the high-dimensional nature of HSI data. We present FOCUS, the first framework that enables reliable and efficient spatial-spectral interpretability for frozen ViTs. FOCUS introduces two core components: class-specific spectral prompts that guide attention toward semantically meaningful wavelength groups, and a learnable [SINK] token trained with an attraction loss to absorb noisy or redundant attention. Together, these designs make it possible to generate stable and interpretable 3D saliency maps and spectral importance curves in a single forward pass, without any gradient backpropagation or backbone modification. FOCUS improves band-level IoU by 15 percent, reduces attention collapse by over 40 percent, and produces saliency results that align closely with expert annotations. With less than 1 percent parameter overhead, our method makes high-resolution ViT interpretability practical for real-world hyperspectral applications, bridging a long-standing gap between black-box modeling and trustworthy HSI decision-making.
Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 33 of its 50 references that are in the map. First posted on arXiv 20 Jul 2025.
Builds on
33 of 50 references.
References · 33
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
- 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 939
- 2023Vision Transformers Need Registerscited by 871
- 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 838
- 2020Quantifying Attention Flow in Transformerscited by 775
- 2021Going deeper with Image Transformerscited by 764
- 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 737
- 2020Transformer Interpretability Beyond Attention Visualizationcited by 627
- 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 582
- 2021DeepViT: Towards Deeper Vision Transformercited by 289
21 more
- 2020Eigen-CAM: Class Activation Map using Principal Componentscited by 189
- 2021Adaptive Token Sampling For Efficient Vision Transformerscited by 183
- 2021SpectralFormer: Rethinking Hyperspectral Image Classification with Transformerscited by 169
- 2020Axiom-based Grad-CAM: Towards Accurate Visualization and Explanation of CNNscited by 151
- 2019Smooth Grad-CAM++: An Enhanced Inference Level Visualization Technique for Deep Convolutional Neural Network Modelscited by 118
- 2022HyperPrompt: Prompt-based Task-Conditioning of Transformerscited by 67
- 2022ProtoPFormer: Concentrating on Prototypical Parts in Vision Transformers for Interpretable Image Recognitioncited by 63
- 2023Zero-TPrune: Zero-Shot Token Pruning through Leveraging of the Attention Graph in Pre-Trained Transformerscited by 62
- 2023Transferable Adversarial Attacks on Vision Transformers with Token Gradient Regularizationcited by 42
- 2022Optimizing Relevance Maps of Vision Transformers Improves Robustnesscited by 41
- 2025Visual Instance-aware Prompt Tuningcited by 29
- 2018HSI-CNN: A Novel Convolution Neural Network for Hyperspectral Imagecited by 15
- 2023A Simple Interpretable Transformer for Fine-Grained Image Classification and Analysiscited by 15
- 2025Visual Variational Autoencoder Prompt Tuningcited by 10
- 2016SANet: Structure-Aware Network for Visual Trackingcited by 9
- 2024HSViT: Horizontally Scalable Vision Transformercited by 6
- 2024SpecFormer: Guarding Vision Transformer Robustness via Maximum Singular Value Penalizationcited by 5
- 2021SSA: Semantic Structure Aware Inference for Weakly Pixel-Wise Dense Predictions without Costcited by 4
- 2023Learning to Mask and Permute Visual Tokens for Vision Transformer Pre-Trainingcited by 3
- 2025EDIT: Enhancing Vision Transformers by Mitigating Attention Sink through an Encoder-Decoder Architecturecited by 2
- 2025Distributed Cross-Channel Hierarchical Aggregation for Foundation Modelscited by 1
Built on by
Citing papers per year: 2025: 4; 2026: 3.
7 papers.
- 2025Prompt-based Adaptation in Large-scale Vision Models: A Surveycited by 21
- 2026Not All Directions Matter: Towards Structured and Task-Aware Low-Rank Model Adaptationcited by 14
- 2026Attention Sink in Transformers: A Survey on Utilization, Interpretation, and Mitigationcited by 11
- 2025CTR-LoRA: Curvature-Aware and Trust-Region Guided Low-Rank Adaptation for Large Language Modelscited by 6
- 2025Self-Supervised Visual Prompting for Cross-Domain Road Damage Detectioncited by 3
- 2026Layer-Specific Prompt Fusion Discovery via Differentiable Search in Vision Foundation Modelscited by 2
- 2025TASAM: Terrain-and-Aware Segment Anything Model for Temporal-Scale Remote Sensing Segmentationcited by 1
- 2026Layer-Specific Prompt Fusion Discovery via Differentiable Search in Vision Foundation Models24 Jun 2026
- 2026Attention Sink in Transformers: A Survey on Utilization, Interpretation, and Mitigation11 Apr 2026
- 2026Not All Directions Matter: Towards Structured and Task-Aware Low-Rank Model Adaptation15 Mar 2026
- 2025Self-Supervised Visual Prompting for Cross-Domain Road Damage Detection16 Nov 2025
- 2025Prompt-based Adaptation in Large-scale Vision Models: A Survey15 Oct 2025
- 2025CTR-LoRA: Curvature-Aware and Trust-Region Guided Low-Rank Adaptation for Large Language Models11 Oct 2025
- 2025TASAM: Terrain-and-Aware Segment Anything Model for Temporal-Scale Remote Sensing Segmentation19 Sep 2025