# Label-Efficient Hyperspectral Image Classification via Spectral FiLM Modulation of Low-Level Pretrained Diffusion Features

arXiv: https://arxiv.org/abs/2512.03430 · HTML: https://mlsanity.com/paper/2512.03430/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 28 of its 39 references that are in the map.
- First posted on arXiv 3 Dec 2025.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 28 of 28)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Label-Efficient Semantic Segmentation with Diffusion Models](https://mlsanity.com/paper/2112.03126/) (2021)
- [Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.04803/) (2023)
- [Unsupervised Domain Adaptation with Residual Transfer Networks](https://mlsanity.com/paper/1602.04433/) (2016)
- [A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence](https://mlsanity.com/paper/2305.15347/) (2023)
- [Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondence](https://mlsanity.com/paper/2305.14334/) (2023)
- [SpectralFormer: Rethinking Hyperspectral Image Classification with Transformers](https://mlsanity.com/paper/2107.02988/) (2021)
- [Pretraining is All You Need for Image-to-Image Translation](https://mlsanity.com/paper/2205.12952/) (2022)
- [Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2312.12470/) (2023)
- [Going Deeper with Contextual CNN for Hyperspectral Image Classification](https://mlsanity.com/paper/1604.03519/) (2016)
- [Self-Supervised Material and Texture Representation Learning for Remote Sensing Tasks](https://mlsanity.com/paper/2112.01715/) (2021)
- [Multimodal Remote Sensing Benchmark Datasets for Land Cover Classification with A Shared and Specific Feature Learning Model](https://mlsanity.com/paper/2105.10196/) (2021)
- [HIR-Diff: Unsupervised Hyperspectral Image Restoration Via Improved Diffusion Models](https://mlsanity.com/paper/2402.15865/) (2024)
- [Spectral Enhanced Rectangle Transformer for Hyperspectral Image Denoising](https://mlsanity.com/paper/2304.00844/) (2023)
- [SpectralDiff: A Generative Framework for Hyperspectral Image Classification with Diffusion Models](https://mlsanity.com/paper/2304.05961/) (2023)
- [DDPM-CD: Denoising Diffusion Probabilistic Models as Feature Extractors for Change Detection](https://mlsanity.com/paper/2206.11892/) (2022)
- [MethaneMapper: Spectral Absorption aware Hyperspectral Transformer for Methane Detection](https://mlsanity.com/paper/2304.02767/) (2023)
- [Diffusion Tuning: Transferring Diffusion Models via Chain of Forgetting](https://mlsanity.com/paper/2406.00773/) (2024)

## Built on by (most cited first, 1 of 1)

- [Synergising Local Geo-Environmental Characteristics with Spatial Context for Enhancing Landslide Susceptibility Mapping](https://mlsanity.com/paper/2608.24956/) (2026)

## Abstract (arXiv)

Hyperspectral imaging (HSI) enables detailed land cover classification, yet low spatial resolution and sparse annotations pose significant challenges. We present a label-efficient framework that leverages spatial features from a frozen diffusion model pretrained on natural images. Our approach extracts low-level representations from high-resolution decoder layers at early denoising timesteps, which transfer effectively to the low-texture structure of HSI. To integrate spectral and spatial information, we introduce a lightweight FiLM-based fusion module that adaptively modulates frozen spatial features using spectral cues, enabling robust multimodal learning under sparse supervision. Experiments on two recent hyperspectral datasets demonstrate that our method outperforms state-of-the-art approaches using only the provided sparse training labels. Ablation studies further highlight the benefits of diffusion-derived features and spectral-aware fusion. Overall, our results indicate that pretrained diffusion models can support domain-agnostic, label-efficient representation learning for remote sensing and broader scientific imaging tasks.