# SPECTRA-VLA: Base-Fine Hyperspectral Token Transfer at Frozen Vision-Language-Action Interfaces

arXiv: https://arxiv.org/abs/2610.19651 · HTML: https://mlsanity.com/paper/2610.19651/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 43 references that are in the map.
- First posted on arXiv 14 Oct 2026.

## Placed in

- Vision-language-action models

## Builds on (most cited first, 30 of 34)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset](https://mlsanity.com/paper/2403.12945/) (2024)
- [AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition](https://mlsanity.com/paper/2205.13535/) (2022)
- [BridgeData V2: A Dataset for Robot Learning at Scale](https://mlsanity.com/paper/2308.12952/) (2023)
- [SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model](https://mlsanity.com/paper/2501.15830/) (2025)
- [SpectralGPT: Spectral Remote Sensing Foundation Model](https://mlsanity.com/paper/2311.07113/) (2023)
- [Multimodal Token Fusion for Vision Transformers](https://mlsanity.com/paper/2204.08721/) (2022)
- [PointVLA: Injecting the 3D World into Vision-Language-Action Models](https://mlsanity.com/paper/2503.07511/) (2025)
- [Neural Plasticity-Inspired Multimodal Foundation Model for Earth Observation](https://mlsanity.com/paper/2403.15356/) (2024)
- [HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model](https://mlsanity.com/paper/2406.11519/) (2024)
- [TacVLA: Contact-Aware Tactile Fusion for Robust Vision-Language-Action Manipulation](https://mlsanity.com/paper/2603.12665/) (2026)
- [HyperFree: A Channel-adaptive and Tuning-free Foundation Model for Hyperspectral Remote Sensing Imagery](https://mlsanity.com/paper/2503.21841/) (2025)
- [HapticVLA: Contact-Rich Manipulation via Vision-Language-Action Model without Inference-Time Tactile Sensing](https://mlsanity.com/paper/2603.15257/) (2026)
- [OmniVLA: Physically-Grounded Multimodal VLA with Unified Multi-Sensor Perception for Robotic Manipulation](https://mlsanity.com/paper/2511.01210/) (2025)
- [PointACT: Vision-Language-Action Models with Multi-Scale Point-Action Interaction](https://mlsanity.com/paper/2605.21414/) (2026)
- [SpectralWaste Dataset: Multimodal Data for Waste Sorting Automation](https://mlsanity.com/paper/2403.18033/) (2024)
- [Hyperspectral Adapter for Semantic Segmentation with Vision Foundation Models](https://mlsanity.com/paper/2509.20107/) (2025)
- [Multimodal Material Classification for Robots using Spectroscopy and High Resolution Texture Imaging](https://mlsanity.com/paper/2004.01160/) (2020)
- [Hyper-Drive: Visible-Short Wave Infrared Hyperspectral Imaging Datasets for Robots in Unstructured Environments](https://mlsanity.com/paper/2308.08058/) (2023)
- [Towards Cross-modal Backward-compatible Representation Learning for Vision-Language Models](https://mlsanity.com/paper/2405.14715/) (2024)
- [HARP-VLA: Human-Robot Aligned Representation Learning for Vision-Language-Action Model](https://mlsanity.com/paper/2605.31234/) (2026)
- [Tactile Modality Fusion for Vision-Language-Action Models](https://mlsanity.com/paper/2603.14604/) (2026)
- [Revisiting Residual Connections: Orthogonal Updates for Stable and Efficient Deep Networks](https://mlsanity.com/paper/2505.11881/) (2025)
- [HyperVision: A Channel-Adaptive Ground-Based Hyperspectral Vision Pre-trained Backbone](https://mlsanity.com/paper/2605.17286/) (2026)
- [MuseVLA: An Adaptive Multimodal Sensing Vision-Language-Action Model for Robotic Manipulation](https://mlsanity.com/paper/2606.17598/) (2026)

## Abstract (arXiv)

Hyperspectral imaging distinguishes RGB-similar objects, but a pretrained vision-language-action model needs a state its frozen decoder can use. SPECTRA-VLA provides a native-width spectral upgrade without retraining the visual encoder or action decoder. A learned continuous residual enriches each target token through its own normalized false-color reference; projection, norm control and validation-calibrated injection regulate the change. Across 86 held-out acquisitions, category balanced accuracy rises from 0.812 to 0.883. With identical residuals and scale, own-token addition improves reuse of an external category-derived probe on frozen decoder logits: 0.701 versus 0.590 with pooled replacement, at similar image-disjoint recognition. Five grouped tests covering the same 86 acquisitions recover within-category spectral shape with R2=0.802/0.337 from continuous and posterior-code states. Category-matched spectral swaps further show that own-token composition approximately doubles the target-specific recovery increment after frozen decoding. Spectral enrichment and receiving-token preservation thus work together at the pretrained interface.