# HyperspectralViTs: General Hyperspectral Models for On-board Remote Sensing

arXiv: https://arxiv.org/abs/2410.17248 · HTML: https://mlsanity.com/paper/2410.17248/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 77 references that are in the map.
- First posted on arXiv 22 Oct 2024.

## Builds on (most cited first, 20 of 20)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [UberNet: Training a `Universal' Convolutional Neural Network for Low-, Mid-, and High-Level Vision using Diverse Datasets and Limited Memory](https://mlsanity.com/paper/1609.02132/) (2016)
- [SpectralFormer: Rethinking Hyperspectral Image Classification with Transformers](https://mlsanity.com/paper/2107.02988/) (2021)
- [HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model](https://mlsanity.com/paper/2406.11519/) (2024)
- [SpectralEarth: Training Hyperspectral Foundation Models at Scale](https://mlsanity.com/paper/2408.08447/) (2024)
- [A review of machine learning in processing remote sensing data for mineral exploration](https://mlsanity.com/paper/2103.07678/) (2021)
- [Unlocking the Use of Raw Multispectral Earth Observation Imagery for Onboard Artificial Intelligence](https://mlsanity.com/paper/2305.11891/) (2023)
- [Dynamically pruning segformer for efficient semantic segmentation](https://mlsanity.com/paper/2111.09499/) (2021)
- [Fast and Accurate Retrieval of Methane Concentration from Imaging Spectrometer Data Using Sparsity Prior](https://mlsanity.com/paper/2003.02978/) (2020)
- [MethaneMapper: Spectral Absorption aware Hyperspectral Transformer for Methane Detection](https://mlsanity.com/paper/2304.02767/) (2023)
- [Fast model inference and training on-board of Satellites](https://mlsanity.com/paper/2307.08700/) (2023)
- [Deep Active Learning in Remote Sensing for data efficient Change Detection](https://mlsanity.com/paper/2008.11201/) (2020)
- [Machine learning based hyperspectral image analysis: A survey](https://mlsanity.com/paper/1802.08701/) (2018)
- [Accelerating Deep Learning Applications in Space](https://mlsanity.com/paper/2007.11089/) (2020)
- [Fast and accurate object detection in high resolution 4K and 8K video using GPUs](https://mlsanity.com/paper/1810.10551/) (2018)
- [Impact of Scene-Specific Enhancement Spectra on Matched Filter Greenhouse Gas Retrievals from Imaging Spectroscopy](https://mlsanity.com/paper/2107.05578/) (2021)
- [Machine Learning for Methane Detection and Quantification from Space - A survey](https://mlsanity.com/paper/2408.15122/) (2024)

## Built on by (most cited first, 11 of 11)

- [Optimizing Methane Detection On Board Satellites: Speed, Accuracy, and Low-Power Solutions for Resource-Constrained Hardware](https://mlsanity.com/paper/2507.01472/) (2025)
- [Operational machine learning for remote spectroscopic detection of CH4 point sources](https://mlsanity.com/paper/2511.07719/) (2025)
- [Multi-Platform Methane Plume Detection via Model and Domain Adaptation](https://mlsanity.com/paper/2506.06348/) (2025)
- [SpectralEarth-FM: Bringing Hyperspectral Imagery into Multimodal Earth Observation Pretraining](https://mlsanity.com/paper/2605.21075/) (2026)
- [Towards Methane Detection Onboard Satellites](https://mlsanity.com/paper/2509.00626/) (2025)
- [Fully Automatic Trace Gas Plume Detection](https://mlsanity.com/paper/2605.03372/) (2026)
- [FLAME: Physics-Guided Neural Operators for Onboard Satellite Methane Detection in Hyperspectral Imagery](https://mlsanity.com/paper/2606.01577/) (2026)
- [Plume Segmentation from MethaneSAT with Cross-Sensor Transfer Learning and Physics-Informed Postprocessing](https://mlsanity.com/paper/2605.24273/) (2026)
- [A Fast Methane Detection Pipeline on Board Satellites Based on Mag1c-SAS and LinkNet](https://mlsanity.com/paper/2606.03675/) (2026)
- [On-board ML for Trace Gas detection in Imaging Spectroscopy data](https://mlsanity.com/paper/2609.04458/) (2026)
- [Methane Detection On Board Satellites from Unorthorectified Imagery](https://mlsanity.com/paper/2609.04906/) (2026)

## Abstract (arXiv)

On-board processing of hyperspectral data with machine learning models would enable unprecedented amount of autonomy for a wide range of tasks, for example methane detection or mineral identification. This can enable early warning system and could allow new capabilities such as automated scheduling across constellations of satellites. Classical methods suffer from high false positive rates and previous deep learning models exhibit prohibitive computational requirements. We propose fast and accurate machine learning architectures which support end-to-end training with data of high spectral dimension without relying on hand-crafted products or spectral band compression preprocessing. We evaluate our models on two tasks related to hyperspectral data processing. With our proposed general architectures, we improve the F1 score of the previous methane detection state-of-the-art models by 27% on a newly created synthetic dataset and by 13% on the previously released large benchmark dataset. We also demonstrate that training models on the synthetic dataset improves performance of models finetuned on the dataset of real events by 6.9% in F1 score in contrast with training from scratch. On a newly created dataset for mineral identification, our models provide 3.5% improvement in the F1 score in contrast to the default versions of the models. With our proposed models we improve the inference speed by 85% in contrast to previous classical and deep learning approaches by removing the dependency on classically computed features. With our architecture, one capture from the EMIT sensor can be processed within 30 seconds on realistic proxy of the ION-SCV 004 satellite.