RoboChemist: Long-Horizon and Safety-Compliant Robotic Chemical Experimentation
On the map: Vision-language-action models → Failure recovery and robustness
Abstract (arXiv)
Robotic chemists promise to both liberate human experts from repetitive tasks and accelerate scientific discovery, yet remain in their infancy. Chemical experiments involve long-horizon procedures over hazardous and deformable substances, where success requires not only task completion but also strict compliance with experimental norms. To address these challenges, we propose textitRoboChemist, a dual-loop framework that integrates Vision-Language Models (VLMs) with Vision-Language-Action (VLA) models. Unlike prior VLM-based systems (e.g., VoxPoser, ReKep) that rely on depth perception and struggle with transparent labware, and existing VLA systems (e.g., RDT, pi0) that lack semantic-level feedback for complex tasks, our method leverages a VLM to serve as (1) a planner to decompose tasks into primitive actions, (2) a visual prompt generator to guide VLA models, and (3) a monitor to assess task success and regulatory compliance. Notably, we introduce a VLA interface that accepts image-based visual targets from the VLM, enabling precise, goal-conditioned control. Our system successfully executes both primitive actions and complete multi-step chemistry protocols. Results show 23.57% higher average success rate and a 0.298 average increase in compliance rate over state-of-the-art VLA baselines, while also demonstrating strong generalization to objects and tasks.
Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Failure recovery and robustness, one of 13 directions in Vision-language-action models. Builds on 70 of its 82 references that are in the map. First posted on arXiv 10 Sep 2025.
Builds on
70 of 82 references.
References · 70
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2023GPT-4 Technical Reportcited by 18,132
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2023Segment Anythingcited by 8,158
- 2023Visual Instruction Tuningcited by 7,792
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2024GPT-4o System Cardcited by 4,467
38 more
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
- 2022Visual Prompt Tuningcited by 1,674
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
- 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
- 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
- 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
- 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
- 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
- 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
- 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
- 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
- 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
- 2023An Embodied Generalist Agent in 3D Worldcited by 355
- 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 344
- 2024TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulationcited by 331
- 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 320
- 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 305
- 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 298
- 2023Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4Vcited by 291
- 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
- 2022Visual Prompting via Image Inpaintingcited by 248
- 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 202
- 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 176
- 2023ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Promptscited by 160
- 2022F-VLM: Open-Vocabulary Object Detection upon Frozen Vision and Language Modelscited by 149
- 2024MOKA: Open-World Robotic Manipulation through Mark-Based Visual Promptingcited by 126
- 2023Explicit Visual Prompting for Low-Level Structure Segmentationscited by 113
- 2024RoboMamba: Efficient Vision-Language-Action Model for Robotic Reasoning and Manipulationcited by 102
- 2024What Matters in Building Vision-Language-Action Models for Generalist Robotscited by 88
- 2025Universal Actions for Enhanced Embodied Foundation Modelscited by 80
- 2025DiffVLA: Vision-Language Guided Diffusion Planning for Autonomous Drivingcited by 64
50 most cited of 70 All 70 (JSON)
Built on by
Citing papers per year: 2025: 1; 2026: 8.
9 papers.
- 2025A Survey on Efficient Vision-Language-Action Modelscited by 34
- 2026Dexora: Open-source VLA for High-DoF Bimanual Dexteritycited by 9
- 2026LIBERO-Safety: A Comprehensive Benchmark for Physical and Semantic Safety in Vision-Language-Action Modelscited by 8
- 2026LABSHIELD: A Multimodal Benchmark for Safety-Critical Reasoning and Planning in Scientific Laboratoriescited by 3
- 2026BioProVLA-Agent: An Affordable, Protocol-Driven, Vision-Enhanced VLA-Enabled Embodied Multi-Agent System with Closed-Loop-Capable Reasoning for Biological Laboratory Manipulationcited by 3
- 2026Predict, Don't Iterate: Efficient Adaptive-Length Infilling for Diffusion Language Modelscited by 3
- 2026LabGuard: Grounding Natural-Language Laboratory Rules into Runtime Guards for Embodied Laboratory Agentscited by 2
- 2026Long-Term Memory for VLA-based Agents in Open-World Task Executioncited by 0
- 2026Security and Privacy in Large-Model-Driven Embodied Agents: Attacks, Defenses, and Future Directionscited by 0
- 2026Predict, Don't Iterate: Efficient Adaptive-Length Infilling for Diffusion Language Models2 Sep 2026
- 2026Security and Privacy in Large-Model-Driven Embodied Agents: Attacks, Defenses, and Future Directions19 Aug 2026
- 2026LabGuard: Grounding Natural-Language Laboratory Rules into Runtime Guards for Embodied Laboratory Agents30 Jun 2026
- 2026LIBERO-Safety: A Comprehensive Benchmark for Physical and Semantic Safety in Vision-Language-Action Models22 Jun 2026
- 2026Dexora: Open-source VLA for High-DoF Bimanual Dexterity18 May 2026
- 2026BioProVLA-Agent: An Affordable, Protocol-Driven, Vision-Enhanced VLA-Enabled Embodied Multi-Agent System with Closed-Loop-Capable Reasoning for Biological Laboratory Manipulation8 May 2026
- 2026Long-Term Memory for VLA-based Agents in Open-World Task Execution17 Apr 2026
- 2026LABSHIELD: A Multimodal Benchmark for Safety-Critical Reasoning and Planning in Scientific Laboratories12 Mar 2026
- 2025A Survey on Efficient Vision-Language-Action Models27 Oct 2025
Similar, not cited
Close in wording, no citation either way
- 2026LabVLA: Grounding Vision-Language-Action Models in Scientific Laboratories
- 2026AtomBridge: Agentic VLA Inference Plugin for Long-Horizon Tasks in Scientific Experiments
- 2026CAPER: Constrained and Procedural Reasoning for Robotic Scientific Experiments
- 2026SAFE-CHEM: Uncertainty-Aware Policy Switching for Robust Robotic Chemistry
- 2025AutoBio: A Simulation and Benchmark for Robotic Automation in Digital Biology Laboratory
- 2026LabRobFail: A Benchmark for Robotic Failure Analysis in Chemical Self-driving Laboratory
- 2026MANGO: Automated Multi-Agent Test Oracle Generation for Vision-Language-Action Models
- 2025TinyChemVL: Advancing Chemical Vision-Language Models via Efficient Visual Token Reduction and Complex Reaction Tasks