Robot Collapse: Supply Chain Backdoor Attacks Against VLM-based Robotic Manipulation

Robotic manipulation policies are increasingly empowered by textitlarge language models (LLMs) and textitvision-language models (VLMs), leveraging their understanding and perception capabilities. Recently, inference-time attacks against robotic manipulation have been extensively studied, yet backdoor attacks targeting model supply chain security in robotic policies remain largely unexplored. To fill this gap, we propose textttTrojanRobot, a backdoor injection framework for model supply chain attack scenarios, which embeds a malicious module into modular robotic policies via backdoor relationships to manipulate the LLM-to-VLM pathway and compromise the system. Our vanilla design instantiates this module as a backdoor-finetuned VLM. To further enhance attack performance, we propose a prime scheme by introducing the concept of textitLVLM-as-a-backdoor, which leverages textitin-context instruction learning (ICIL) to steer textitlarge vision-language model (LVLM) behavior through backdoored system prompts. Moreover, we develop three types of prime attacks, textitpermutation, textitstagnation, and textitintentional, achieving flexible backdoor attack effects. Extensive physical-world and simulator experiments on 18 real-world manipulation tasks and 4 VLMs verify the superiority of proposed textttTrojanRobot