Safe-Night VLA: Seeing the Unseen via Thermal-Perceptive Vision-Language-Action Models for Safety-Critical Manipulation
On the map: Vision-language-action models → Safety and adversarial attacks
Abstract (arXiv)
Current Vision-Language-Action (VLA) models rely primarily on RGB perception, preventing them from capturing modalities such as thermal signals that are imperceptible to conventional visual sensors. Moreover, end-to-end generative policies lack explicit safety constraints, making them fragile when encountering obstacles and novel scenarios outside the training distribution. To address these limitations, we propose Safe-Night VLA, a multimodal manipulation framework that enables robots to see the unseen while enforcing rigorous safety constraints for thermal-aware manipulation in unstructured environments. Specifically, Safe-Night VLA integrates long-wave infrared thermal perception into a pre-trained vision-language backbone, enabling semantic reasoning grounded in thermodynamic properties. To ensure safe execution under out-of-distribution conditions, we incorporate a safety filter via control barrier functions, which provide deterministic workspace constraint enforcement during policy execution. We validate our framework through real-world experiments on a Franka manipulator, introducing a novel evaluation paradigm featuring temperature-conditioned manipulation, subsurface target localization, and reflection disambiguation, while maintaining constrained execution at inference time. Results demonstrate that Safe-Night VLA outperforms RGB-only baselines and provide empirical evidence that foundation models can effectively leverage non-visible physical modalities for robust manipulation.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Safety and adversarial attacks, one of 13 directions in Vision-language-action models. Builds on 25 of its 37 references that are in the map. First posted on arXiv 5 Mar 2026.
Builds on
25 of 37 references.
Closest ancestors · 1
Same direction, newest first
Other references · 24
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
- 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
- 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 320
- 2025π*0.6: a VLA That Learns From Experiencecited by 280
- 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 261
- 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 167
- 2019ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imagingcited by 145
- 2023Visual Prompt Multi-Modal Trackingcited by 141
- 2023SafeDiffuser: Safe Planning with Diffusion Probabilistic Modelscited by 69
12 more
- 2025Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Modelscited by 62
- 2025OmniVLA: An Omni-Modal Vision-Language-Action Model for Robot Navigationcited by 34
- 2023Robotic Perception of Transparent Objects: A Reviewcited by 28
- 2024ReVLA: Reverting Visual Domain Limitation of Robotic Foundation Modelscited by 26
- 2025STARE-VLA: Progressive Stage-Aware Reinforcement for Fine-Tuning Vision-Language-Action Modelscited by 24
- 2023Audio-Visual Efficient Conformer for Robust Speech Recognitioncited by 22
- 2023ShapeClipper: Scalable 3D Shape Learning from Single-View Images via Geometric and CLIP-based Consistencycited by 16
- 2023TRansPose: Large-Scale Multispectral Dataset for Transparent Objectcited by 15
- 2024Toward a Holistic Evaluation of Robustness in CLIP Modelscited by 13
- 2025SafeFlow: Safe Robot Motion Planning with Flow Matching via Control Barrier Functionscited by 12
- 2025OmniVLA: Physically-Grounded Multimodal VLA with Unified Multi-Sensor Perception for Robotic Manipulationcited by 7
- 2025UniConFlow: A Unified Constrained Flow-Matching Framework for Certified Motion Planningcited by 4
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026Vision-Language-Action Safety: Threats, Challenges, Evaluations, and Mechanismscited by 12
- 2026SafeDojo: Safe Reinforcement Learning for VLA via Interactive World Modelcited by 2
- 2026Security of Foundation-Model-Powered Embodied Agents: Attack Surfaces, Attacks, Defenses, and Evaluationcited by 0
- 2026Security and Privacy in Large-Model-Driven Embodied Agents: Attacks, Defenses, and Future Directionscited by 0
- 2026RawVLA: Embodied Neural Image Signal Processor For Robotic Manipulationcited by 0
- 2026RawVLA: Embodied Neural Image Signal Processor For Robotic Manipulation29 Sep 2026
- 2026Security and Privacy in Large-Model-Driven Embodied Agents: Attacks, Defenses, and Future Directions19 Aug 2026
- 2026Security of Foundation-Model-Powered Embodied Agents: Attack Surfaces, Attacks, Defenses, and Evaluation17 Aug 2026
- 2026SafeDojo: Safe Reinforcement Learning for VLA via Interactive World Model15 Jun 2026
- 2026Vision-Language-Action Safety: Threats, Challenges, Evaluations, and Mechanisms26 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2026ThermoAct:Thermal-Aware Vision-Language-Action Models for Robotic Perception and Decision-Making
- 2026SafeStage: Evaluating Safety Before, During, and After Vision-Language-Conditioned Robot Manipulation ↗
- 2026WBAG: A Whole-Body and Attached-Geometry Safety Framework for Vision-Language-Action Manipulation
- 2026VLPSA: Vision-Language-Poisson-Safe Actions for Full-Body Safety of Learned Policies
- 2025VLSA: Vision-Language-Action Models with Plug-and-Play Safety Constraint Layer
- 2026SafeVLA-Bench: A Benchmark for the Success-Safety Gap in Vision-Language-Action Models
- 2026ForesightSafety-VLA: A Unified Diagnostic Safety Benchmark for Vision-Language-Action Models
- 2026Event-VLA: Action-Conditioned Event Fusion for Robust Vision-Language-Action Model