# Turning spectra into images improves plant trait retrieval with 2D-CNNs

arXiv: https://arxiv.org/abs/2608.16661 · HTML: https://mlsanity.com/paper/2608.16661/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 68 references that are in the map.
- First posted on arXiv 17 Aug 2026.

## Builds on (most cited first, 9 of 9)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Imaging Time-Series to Improve Classification and Imputation](https://mlsanity.com/paper/1506.00327/) (2015)
- [GreenHyperSpectra: A multi-source hyperspectral dataset for global vegetation trait prediction](https://mlsanity.com/paper/2507.06806/) (2025)
- [TerraMAE: Learning Spatial-Spectral Representations from Hyperspectral Earth Observation Data via Adaptive Masked Autoencoders](https://mlsanity.com/paper/2508.07020/) (2025)

## Abstract (arXiv)

Hyperspectral reflectance spectroscopy enables non-destructive estimation of plant functional traits, yet current deep learning approaches process spectra as one-dimensional sequences, which limits how they capture long-range inter-band dependencies. We asked whether transforming 1D spectra into 2D image representations improves multi-trait prediction with convolutional neural networks (CNN). We compared nine transformations using EfficientNet-B0 on the GreenHyperSpectra dataset (7,897 labeled spectra, eight traits, 400-2450 nm), benchmarked against published 1D CNN results on the same split. Trained from scratch, the simplest transformation, a direct Reshape of the spectrum into a 2D grid, performed best (R2 = 0.684 ± 0.001) and improved on the state-of-the-art 1D baseline (R2 = 0.587, +0.097). We then pretrained a 2D masked autoencoder (MAE-2D) on 139,000 unlabeled spectral images. Linear probing, which freezes the encoder and trains only a multilayer perceptron head, reached R2 = 0.646 and exceeded every 1D self-supervised counterpart, including the fine-tuned MAE-1D (R2 = 0.641). Under cross-dataset evaluation all models lost most of their accuracy and none beat the 1D baseline significantly. To identify which wavelengths drive each prediction, we applied Integrated Gradients and Grad-CAM and unfolded band importance back to the spectral axis. Protein (r = 0.45) and leaf water (r = 0.33) agreed with sensitivities simulated by the PROSAIL radiative-transfer model, while carotenoids (r = 0.06) and leaf area index (r = -0.11) did not, showing that the model reads established leaf chemistry for traits with sharp absorption features. The representational advantage of 2D spectral images, rather than architectural complexity or ImageNet pretraining, drives the gain over 1D approaches.