# LABSHIELD: A Multimodal Benchmark for Safety-Critical Reasoning and Planning in Scientific Laboratories arXiv: https://arxiv.org/abs/2603.11987 ยท HTML: https://mlsanity.com/paper/2603.11987/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. - Builds on 45 of its 105 references that are in the map. - First posted on arXiv 12 Mar 2026. ## Placed in - LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/) ## Builds on (most cited first, 30 of 45) - [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024) - [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025) - [RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation](https://mlsanity.com/paper/2506.18088/) (2025) - [A Survey on Vision-Language-Action Models for Embodied AI](https://mlsanity.com/paper/2405.14093/) (2024) - [SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models](https://mlsanity.com/paper/2307.10635/) (2023) - [EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents](https://mlsanity.com/paper/2502.09560/) (2025) - [VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Tasks](https://mlsanity.com/paper/2412.18194/) (2024) - [Agent AI: Surveying the Horizons of Multimodal Interaction](https://mlsanity.com/paper/2401.03568/) (2024) - [ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning](https://mlsanity.com/paper/2503.22738/) (2025) - [SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents](https://mlsanity.com/paper/2412.13178/) (2024) - [RoboBrain 2.0 Technical Report](https://mlsanity.com/paper/2507.02029/) (2025) - [Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challenges](https://mlsanity.com/paper/2505.04769/) (2025) - [Towards Synergistic, Generalized, and Efficient Dual-System for Robotic Manipulation](https://mlsanity.com/paper/2410.08001/) (2024) - [SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learning](https://mlsanity.com/paper/2503.03480/) (2025) - [EgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level Planning](https://mlsanity.com/paper/2312.06722/) (2023) - [Whoever Started the Interference Should End It: Guiding Data-Free Model Merging via Task Vectors](https://mlsanity.com/paper/2503.08099/) (2025) - [Embodied AI Agents: Modeling the World](https://mlsanity.com/paper/2506.22355/) (2025) - [EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agents](https://mlsanity.com/paper/2501.11858/) (2025) - [IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Tasks](https://mlsanity.com/paper/2506.16402/) (2025) - [Multi-agent Embodied AI: Advances and Future Directions](https://mlsanity.com/paper/2505.05108/) (2025) - [VidEgoThink: Assessing Egocentric Video Understanding Capabilities for Embodied AI](https://mlsanity.com/paper/2410.11623/) (2024) - [Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attacks](https://mlsanity.com/paper/2502.13175/) (2025) - [VLSA: Vision-Language-Action Models with Plug-and-Play Safety Constraint Layer](https://mlsanity.com/paper/2512.11891/) (2025) - [Fast-in-Slow: A Dual-System Foundation Model Unifying Fast Manipulation within Slow Reasoning](https://mlsanity.com/paper/2506.01953/) (2025) - [VLA-Arena: An Open-Source Framework for Benchmarking Vision-Language-Action Models](https://mlsanity.com/paper/2512.22539/) (2025) - [A Framework for Benchmarking and Aligning Task-Planning Safety in LLM-Based Embodied Agents](https://mlsanity.com/paper/2504.14650/) (2025) - [RoboChallenge: Large-scale Real-robot Evaluation of Embodied Policies](https://mlsanity.com/paper/2510.17950/) (2025) - [LabSafety Bench: Benchmarking LLMs on Safety Issues in Scientific Labs](https://mlsanity.com/paper/2410.14182/) (2024) - [Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Brain](https://mlsanity.com/paper/2510.17801/) (2025) - [Multi-Task Model Merging via Adaptive Weight Disentanglement](https://mlsanity.com/paper/2411.18729/) (2024) ## Built on by (most cited first, 3 of 3) - [Efficient-WAM: A 1B-Parameter World-Action Model with Low-Cost Future Imagination](https://mlsanity.com/paper/2606.10040/) (2026) - [LabGuard: Grounding Natural-Language Laboratory Rules into Runtime Guards for Embodied Laboratory Agents](https://mlsanity.com/paper/2606.31045/) (2026) - [Visual Compliance via Executable Safety Rule Entailment](https://mlsanity.com/paper/2609.18328/) (2026) ## Abstract (arXiv) Artificial intelligence is increasingly catalyzing scientific automation, with multimodal large language model (MLLM) agents evolving from lab assistants into self-driving lab operators. This transition imposes stringent safety requirements on laboratory environments, where fragile glassware, hazardous substances, and high-precision laboratory equipment render planning errors or misinterpreted risks potentially irreversible. However, the safety awareness and decision-making reliability of embodied agents in such high-stakes settings remain insufficiently defined and evaluated. To bridge this gap, we introduce LABSHIELD, a realistic multi-view benchmark designed to assess MLLMs in hazard identification and safety-critical reasoning. Grounded in U.S. Occupational Safety and Health Administration (OSHA) standards and the Globally Harmonized System (GHS), LABSHIELD establishes a rigorous safety taxonomy spanning 164 operational tasks with diverse manipulation complexities and risk profiles. We evaluate 20 proprietary models, 9 open-source models, and 3 embodied models under a dual-track evaluation framework. Our results reveal a systematic gap between general-domain MCQ accuracy and Semi-open QA safety performance, with models exhibiting an average drop of 32.0% in professional laboratory scenarios, particularly in hazard interpretation and safety-aware planning. These findings underscore the urgent necessity for safety-centric reasoning frameworks to ensure reliable autonomous scientific experimentation in embodied laboratory contexts. The full dataset will be released soon.