# HyperVision: A Channel-Adaptive Ground-Based Hyperspectral Vision Pre-trained Backbone

arXiv: https://arxiv.org/abs/2605.17286 · HTML: https://mlsanity.com/paper/2605.17286/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 56 references that are in the map.
- First posted on arXiv 17 May 2026.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [PVT v2: Improved Baselines with Pyramid Vision Transformer](https://mlsanity.com/paper/2106.13797/) (2021)
- [U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection](https://mlsanity.com/paper/2005.09007/) (2020)
- [SpectralGPT: Spectral Remote Sensing Foundation Model](https://mlsanity.com/paper/2311.07113/) (2023)
- [Visual Prompt Multi-Modal Tracking](https://mlsanity.com/paper/2303.10826/) (2023)
- [Material Based Object Tracking in Hyperspectral Videos: Benchmark and Algorithms](https://mlsanity.com/paper/1812.04179/) (2018)
- [Context-Guided Spatial Feature Reconstruction for Efficient Semantic Segmentation](https://mlsanity.com/paper/2405.06228/) (2024)
- [Hyperspectral Image Dataset for Benchmarking on Salient Object Detection](https://mlsanity.com/paper/1806.11314/) (2018)
- [BA-SAM: Scalable Bias-Mode Attention Mask for Segment Anything Model](https://mlsanity.com/paper/2401.02317/) (2024)
- [Measuring the Ripeness of Fruit with Hyperspectral Imaging and Deep Learning](https://mlsanity.com/paper/2104.09808/) (2021)
- [HS3-Bench: A Benchmark and Strong Baseline for Hyperspectral Semantic Segmentation in Driving Scenarios](https://mlsanity.com/paper/2409.11205/) (2024)
- [HSI-Drive v2.0: More Data for New Challenges in Scene Understanding for Autonomous Driving](https://mlsanity.com/paper/2411.17530/) (2024)
- [Spectrum-driven Mixed-frequency Network for Hyperspectral Salient Object Detection](https://mlsanity.com/paper/2312.01060/) (2023)
- [A Hyperspectral and RGB Dataset for Building Facade Segmentation](https://mlsanity.com/paper/2212.02749/) (2022)
- [Hyperspectral Adapter for Semantic Segmentation with Vision Foundation Models](https://mlsanity.com/paper/2509.20107/) (2025)
- [Hyperspectral City V1.0 Dataset and Benchmark](https://mlsanity.com/paper/1907.10270/) (2019)
- [Hyper-Drive: Visible-Short Wave Infrared Hyperspectral Imaging Datasets for Robots in Unstructured Environments](https://mlsanity.com/paper/2308.08058/) (2023)
- [A Dataset for Evaluating Blood Detection in Hyperspectral Images](https://mlsanity.com/paper/2008.10254/) (2020)

## Built on by (most cited first, 2 of 2)

- [Seeing the Unseen: Camouflaged Object Detection Beyond the Visible Spectrum](https://mlsanity.com/paper/2608.30355/) (2026)
- [SPECTRA-VLA: Base-Fine Hyperspectral Token Transfer at Frozen Vision-Language-Action Interfaces](https://mlsanity.com/paper/2610.19651/) (2026)

## Abstract (arXiv)

While hyperspectral imaging provides rich spatial-spectral information across hundreds of narrow wavelength bands for precise material identification, ground-based hyperspectral pre-trained backbones remain absent, constrained by varying spectral configurations across sensors, limited annotations and heterogeneous labeling schemes, and the limited scale and scene diversity of existing datasets. To address these challenges and enable universal perception, we propose HyperVision, the first ground-based hyperspectral pre-trained backbone. First, to handle varying spectral configurations, HyperVision adopts a channel-adaptive dynamic embedding mechanism to map heterogeneous inputs into a unified token space. Second, we develop an unsupervised representation learning framework. Specifically, to address limited annotations and heterogeneous labeling schemes, a multi-source pseudo-labeling method is introduced to fuse spatial structures from SAM2 and fine-grained spectral material information from HyperFree. Furthermore, to enrich scene diversity and compensate for limited dataset scale, a cross-modal knowledge distillation mechanism is utilized to transfer rich semantic representations from a pre-trained RGB vision model to our backbone. Pre-trained on a collection of 15k images from 26 diverse ground-based datasets, HyperVision demonstrates exceptional generalization. Requiring only efficient head-only adaptation without adjusting backbone parameters, it outperforms state-of-the-art task-specific methods across three downstream tasks under varying sensor configurations, yielding up to a 16.3% relative improvement in hyperspectral semantic segmentation AccM, a 2.1% relative gain in object tracking AUC, and a 35.5% reduction in salient object detection MAE. The source code and pre-trained models are available at https://github.com/lronkitty/HyperVision.