# End-to-End Unmixing with Material Prompts for Hyperspectral Object Tracking

arXiv: https://arxiv.org/abs/2605.20569 · HTML: https://mlsanity.com/paper/2605.20569/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 54 references that are in the map.
- First posted on arXiv 20 May 2026.

## Builds on (most cited first, 18 of 18)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks](https://mlsanity.com/paper/1812.11703/) (2018)
- [Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework](https://mlsanity.com/paper/2203.11991/) (2022)
- [Visual Prompt Multi-Modal Tracking](https://mlsanity.com/paper/2303.10826/) (2023)
- [End-to-end representation learning for Correlation Filter based tracking](https://mlsanity.com/paper/1704.06036/) (2017)
- [Autoregressive Queries for Adaptive Tracking with Spatio-TemporalTransformers](https://mlsanity.com/paper/2403.10574/) (2024)
- [Multi-Modal Fusion for End-to-End RGB-T Tracking](https://mlsanity.com/paper/1908.11714/) (2019)
- [ARTrackV2: Prompting Autoregressive Tracker Where to Look and How to Describe](https://mlsanity.com/paper/2312.17133/) (2023)
- [Explicit Visual Prompts for Visual Object Tracking](https://mlsanity.com/paper/2401.03142/) (2024)
- [RGBD1K: A Large-scale Dataset and Benchmark for RGB-D Object Tracking](https://mlsanity.com/paper/2208.09787/) (2022)
- [Material Based Object Tracking in Hyperspectral Videos: Benchmark and Algorithms](https://mlsanity.com/paper/1812.04179/) (2018)
- [Separable Self and Mixed Attention Transformers for Efficient Object Tracking](https://mlsanity.com/paper/2309.03979/) (2023)
- [Towards Real-World Visual Tracking with Temporal Contexts](https://mlsanity.com/paper/2308.10330/) (2023)
- [TrackNetV4: Enhancing Fast Sports Object Tracking with Motion Attention Maps](https://mlsanity.com/paper/2409.14543/) (2024)
- [Improving Visual Object Tracking through Visual Prompting](https://mlsanity.com/paper/2409.18901/) (2024)
- [Hy-Tracker: A Novel Framework for Enhancing Efficiency and Accuracy of Object Tracking in Hyperspectral Videos](https://mlsanity.com/paper/2311.18199/) (2023)
- [Object Tracking in Hyperspectral Videos with Convolutional Features and Kernelized Correlation Filter](https://mlsanity.com/paper/1810.11819/) (2018)

## Built on by (most cited first, 1 of 1)

- [MSP-Net: Manifold-Guided Spectral Prompt Network for Hyperspectral Object Tracking](https://mlsanity.com/paper/2608.09575/) (2026)

## Abstract (arXiv)

Hyperspectral imagery encodes rich material properties that can improve tracking robustness under appearance ambiguity, illumination change, and background clutter. However, due to the limited availability of hyperspectral video data, many existing methods adapt pretrained RGB trackers via spatial or channel fusion strategies, largely neglecting the intrinsic material information in hyperspectral imagery. Moreover, the few material-aware approaches typically rely on external spectral unmixing pipelines that are decoupled from the tracking objective, limiting effective optimization of material representations for target localization. To address these limitations, we formulate hyperspectral object tracking as a joint optimization problem of material decomposition and target localization, coupling the two tasks via a weighted target-oriented unmixing loss that explicitly aligns material representations with localization accuracy. Specifically, we propose a material representation decomposition module for deep learning-based spectral unmixing with adaptive frequency decomposition. Building on the decomposed material representations, we further introduce a dual-branch wavelet-enhanced material prompt module that learns low- and high-frequency material prompts through efficient spatial-material interactions in the frequency domain. The framework is model-agnostic and can be seamlessly generalized to different unmixing backbones. Extensive experiments on standard hyperspectral tracking benchmarks demonstrate state-of-the-art performance and validate the effectiveness of the proposed end-to-end material-aware tracking framework. Code is available at https://github.com/han030927/E2EMPT.