mlsanityData to 14 Oct 2026

SPECTRA-VLA: Base-Fine Hyperspectral Token Transfer at Frozen Vision-Language-Action Interfaces

arXiv 2610.19651 ↗First posted 14 Oct 2026cs.CVcited by 0

New in Vision-language-action models; no direction yet. Vision-language-action models

Abstract (arXiv)

Hyperspectral imaging distinguishes RGB-similar objects, but a pretrained vision-language-action model needs a state its frozen decoder can use. SPECTRA-VLA provides a native-width spectral upgrade without retraining the visual encoder or action decoder. A learned continuous residual enriches each target token through its own normalized false-color reference; projection, norm control and validation-calibrated injection regulate the change. Across 86 held-out acquisitions, category balanced accuracy rises from 0.812 to 0.883. With identical residuals and scale, own-token addition improves reuse of an external category-derived probe on frozen decoder logits: 0.701 versus 0.590 with pooled replacement, at similar image-disjoint recognition. Five grouped tests covering the same 86 acquisitions recover within-category spectral shape with R2=0.802/0.337 from continuous and posterior-code states. Category-matched spectral swaps further show that own-token composition approximately doubles the target-specific recovery increment after frozen decoding. Spectral enrichment and receiving-token preservation thus work together at the pretrained interface.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 43 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

34 of 43 references.

References · 34

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  3. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  4. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  5. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  6. 2022Visual Prompt Tuningcited by 1,681
  7. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
  8. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 719
  9. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 657
  10. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 519
  11. 2023SpectralGPT: Spectral Remote Sensing Foundation Modelcited by 252
  12. 2022Multimodal Token Fusion for Vision Transformerscited by 136
22 more
  1. 2025PointVLA: Injecting the 3D World into Vision-Language-Action Modelscited by 105
  2. 2024Neural Plasticity-Inspired Multimodal Foundation Model for Earth Observationcited by 104
  3. 2024HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Modelcited by 73
  4. 2026TacVLA: Contact-Aware Tactile Fusion for Robust Vision-Language-Action Manipulationcited by 19
  5. 2025HyperFree: A Channel-adaptive and Tuning-free Foundation Model for Hyperspectral Remote Sensing Imagerycited by 17
  6. 2026HapticVLA: Contact-Rich Manipulation via Vision-Language-Action Model without Inference-Time Tactile Sensingcited by 10
  7. 2025OmniVLA: Physically-Grounded Multimodal VLA with Unified Multi-Sensor Perception for Robotic Manipulationcited by 9
  8. 2026PointACT: Vision-Language-Action Models with Multi-Scale Point-Action Interactioncited by 8
  9. 2024SpectralWaste Dataset: Multimodal Data for Waste Sorting Automationcited by 7
  10. 2025Hyperspectral Adapter for Semantic Segmentation with Vision Foundation Modelscited by 7
  11. 2020Multimodal Material Classification for Robots using Spectroscopy and High Resolution Texture Imagingcited by 5
  12. 2023Hyper-Drive: Visible-Short Wave Infrared Hyperspectral Imaging Datasets for Robots in Unstructured Environmentscited by 5
  13. 2024Towards Cross-modal Backward-compatible Representation Learning for Vision-Language Modelscited by 5
  14. 2026HARP-VLA: Human-Robot Aligned Representation Learning for Vision-Language-Action Modelcited by 5
  15. 2026Tactile Modality Fusion for Vision-Language-Action Modelscited by 4
  16. 2025Revisiting Residual Connections: Orthogonal Updates for Stable and Efficient Deep Networkscited by 2
  17. 2026HyperVision: A Channel-Adaptive Ground-Based Hyperspectral Vision Pre-trained Backbonecited by 2
  18. 2026MuseVLA: An Adaptive Multimodal Sensing Vision-Language-Action Model for Robotic Manipulationcited by 2
  19. 2025UniDiff: Parameter-Efficient Adaptation of Diffusion Models for Land Cover Classification with Multi-Modal Remotely Sensed Imagery and Sparse Annotationscited by 1
  20. 2025A Hyperspectral Imaging Guided Robotic Grasping Systemcited by 1
  21. 2026Bidirectional Cross-Attention Fusion of High-Resolution RGB and Low-Resolution Hyperspectral Inputs for Multimodal Semantic Segmentationcited by 1
  22. 2026TAP-VLA: Tactile Annotation Prompting for Vision Language Action Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond RGB: Benchmarking and Enhancing MLLMs for Hyperspectral Image Understanding via Training-Free Reasoning Framework
  2. 2026Beyond Compression: Quantifying Spectral Accessibility in Vision Representations
  3. 2025HYDRA: HYbrid knowledge Distillation and spectral Reconstruction Algorithm for high channel hyperspectral camera applications
  4. 2026SpectraDINO: Modality-Conditioned Adaptation of RGB Vision Foundation Models Across Infrared Bands
  5. 2025From Pixels and Words to Waves: A Unified Framework for Spectral Dictionary vLLMs
  6. 2026Canonical Color as a Lens into Concept Decodability in Vision Encoders and VLMs
  7. 2026Vision-Language Guided Hyperspectral Object Tracking via Semantics Fusion and Contextual Template Updating
  8. 2024ColorMamba: Towards High-quality NIR-to-RGB Spectral Translation with Mamba