# Material Based Object Tracking in Hyperspectral Videos: Benchmark and Algorithms

arXiv: https://arxiv.org/abs/1812.04179 · HTML: https://mlsanity.com/paper/1812.04179/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2018.

## Built on by (most cited first, 30 of 36)

- [HSR-Diff:Hyperspectral Image Super-Resolution via Conditional Diffusion Models](https://mlsanity.com/paper/2306.12085/) (2023)
- [SSUMamba: Spatial-Spectral Selective State Space Model for Hyperspectral Image Denoising](https://mlsanity.com/paper/2405.01726/) (2024)
- [Decoupled-and-Coupled Networks: Self-Supervised Hyperspectral Image Super-Resolution with Subpixel Fusion](https://mlsanity.com/paper/2205.03742/) (2022)
- [Implicit Neural Representation Learning for Hyperspectral Image Super-Resolution](https://mlsanity.com/paper/2112.10541/) (2021)
- [HyperFree: A Channel-adaptive and Tuning-free Foundation Model for Hyperspectral Remote Sensing Imagery](https://mlsanity.com/paper/2503.21841/) (2025)
- [MUST: The First Dataset and Unified Framework for Multispectral UAV Single Object Tracking](https://mlsanity.com/paper/2503.17699/) (2025)
- [Object Detection in Hyperspectral Image via Unified Spectral-Spatial Feature Aggregation](https://mlsanity.com/paper/2306.08370/) (2023)
- [Deep Amended Gradient Descent for Efficient Spectral Reconstruction from Single RGB Images](https://mlsanity.com/paper/2108.05547/) (2021)
- [Semantic-embedded Unsupervised Spectral Reconstruction from Single RGB Images in the Wild](https://mlsanity.com/paper/2108.06659/) (2021)
- [Hy-Tracker: A Novel Framework for Enhancing Efficiency and Accuracy of Object Tracking in Hyperspectral Videos](https://mlsanity.com/paper/2311.18199/) (2023)
- [Hyperspectral Image Denoising via Spatial-Spectral Recurrent Transformer](https://mlsanity.com/paper/2401.03885/) (2023)
- [Unsupervised Spectral Demosaicing with Lightweight Spectral Attention Networks](https://mlsanity.com/paper/2307.01990/) (2023)
- [BihoT: A Large-Scale Dataset and Benchmark for Hyperspectral Camouflaged Object Tracking](https://mlsanity.com/paper/2408.12232/) (2024)
- [Vision-Based Anti Unmanned Aerial Technology: Opportunities and Challenges](https://mlsanity.com/paper/2507.10006/) (2025)
- [Continuous Spectral Reconstruction from RGB Images via Implicit Neural Representation](https://mlsanity.com/paper/2112.13003/) (2021)
- [SSF-Net: Spatial-Spectral Fusion Network with Spectral Angle Awareness for Hyperspectral Object Tracking](https://mlsanity.com/paper/2403.05852/) (2024)
- [SST-ReversibleNet: Reversible-prior-based Spectral-Spatial Transformer for Efficient Hyperspectral Image Reconstruction](https://mlsanity.com/paper/2305.04054/) (2023)
- [SSIF: Learning Continuous Image Representation for Spatial-Spectral Super-Resolution](https://mlsanity.com/paper/2310.00413/) (2023)
- [A Deep Learning Approach for Pixel-level Material Classification via Hyperspectral Imaging](https://mlsanity.com/paper/2409.13498/) (2024)
- [Hyperspectral Mamba for Hyperspectral Object Tracking](https://mlsanity.com/paper/2509.08265/) (2025)
- [HyperVision: A Channel-Adaptive Ground-Based Hyperspectral Vision Pre-trained Backbone](https://mlsanity.com/paper/2605.17286/) (2026)
- [HHTrack: Hyperspectral Object Tracking Using Hybrid Attention](https://mlsanity.com/paper/2308.07016/) (2023)
- [SUIT: Spatial-Spectral Union-Intersection Interaction Network for Hyperspectral Object Tracking](https://mlsanity.com/paper/2508.07250/) (2025)
- [Hyperspectral Imaging](https://mlsanity.com/paper/2508.08107/) (2025)
- [End-to-End Unmixing with Material Prompts for Hyperspectral Object Tracking](https://mlsanity.com/paper/2605.20569/) (2026)
- [Hyperspectral Adapter for Object Tracking based on Hyperspectral Video](https://mlsanity.com/paper/2503.22199/) (2025)
- [A Deep Unfolding Framework for Diffractive Snapshot Spectral Imaging](https://mlsanity.com/paper/2507.04622/) (2025)
- [A Copula-Guided Temporal Dependency Method for Multitemporal Hyperspectral Images Unmixing](https://mlsanity.com/paper/2509.11096/) (2025)
- [MMOT: The First Challenging Benchmark for Drone-based Multispectral Multi-Object Tracking](https://mlsanity.com/paper/2510.12565/) (2025)
- [Exploring Spatiotemporal Feature Propagation for Video-Level Compressive Spectral Reconstruction: Dataset, Model and Benchmark](https://mlsanity.com/paper/2603.00611/) (2026)

## Abstract (arXiv)

Traditional color images only depict color intensities in red, green and blue channels, often making object trackers fail in challenging scenarios, e.g., background clutter and rapid changes of target appearance. Alternatively, material information of targets contained in a large amount of bands of hyperspectral images (HSI) is more robust to these difficult conditions. In this paper, we conduct a comprehensive study on how material information can be utilized to boost object tracking from three aspects: benchmark dataset, material feature representation and material based tracking. In terms of benchmark, we construct a dataset of fully-annotated videos, which contain both hyperspectral and color sequences of the same scene. Material information is represented by spectral-spatial histogram of multidimensional gradient, which describes the 3D local spectral-spatial structure in an HSI, and fractional abundances of constituted material components which encode the underlying material distribution. These two types of features are embedded into correlation filters, yielding material based tracking. Experimental results on the collected benchmark dataset show the potentials and advantages of material based object tracking.