# DRIFT: Derailing Denoising Trajectories of Flow-Matching VLAs with Adversarial Patch Attack

arXiv: https://arxiv.org/abs/2608.03207 · HTML: https://mlsanity.com/paper/2608.03207/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Efficient VLA inference and adaptation, one of 13 directions in Vision-language-action models.
- Builds on 21 of its 21 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Placed in

- Vision-language-action models / [Efficient VLA inference and adaptation](https://mlsanity.com/vla/efficient-vla-inference-and-adaptation/)
- Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/)

## Builds on (most cited first, 21 of 21)

- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [Octo: An Open-Source Generalist Robot Policy](https://mlsanity.com/paper/2405.12213/) (2024)
- [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023)
- [A study of the effect of JPG compression on adversarial images](https://mlsanity.com/paper/1608.00853/) (2016)
- [Keeping the Bad Guys Out: Protecting and Vaccinating Deep Learning with JPEG Compression](https://mlsanity.com/paper/1705.02900/) (2017)
- [Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Robotics](https://mlsanity.com/paper/2411.13587/) (2024)
- [BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimization](https://mlsanity.com/paper/2505.16640/) (2025)
- [Adversarial Attacks on Robotic Vision Language Action Models](https://mlsanity.com/paper/2506.03350/) (2025)
- [RobustVLA: On Robustness of Vision-Language-Action Model against Multi-Modal Perturbations](https://mlsanity.com/paper/2510.00037/) (2025)
- [Model-agnostic Adversarial Attack and Defense for Vision-Language-Action Models](https://mlsanity.com/paper/2510.13237/) (2025)
- [When Alignment Fails: Multimodal Adversarial Attacks on Vision-Language-Action Models](https://mlsanity.com/paper/2511.16203/) (2025)
- [Diffusion Policy Attacker: Crafting Adversarial Attacks for Diffusion-based Policies](https://mlsanity.com/paper/2405.19424/) (2024)
- [Goal-oriented Backdoor Attack against Vision-Language-Action Models via Physical Objects](https://mlsanity.com/paper/2510.09269/) (2025)
- [When Robots Obey the Patch: Universal Transferable Patch Attacks on Vision-Language-Action Models](https://mlsanity.com/paper/2511.21192/) (2025)
- [TRAP: Hijacking VLA CoT-Reasoning via Adversarial Patches](https://mlsanity.com/paper/2603.23117/) (2026)
- [FlowHijack: A Dynamics-Aware Backdoor Attack on Flow-Matching Vision-Language-Action Models](https://mlsanity.com/paper/2604.09651/) (2026)

## Built on by (most cited first, 1 of 1)

- [Typographic Attack Against VLM-based AI-generated Image Detection](https://mlsanity.com/paper/2609.39662/) (2026)

## Abstract (arXiv)

Flow-matching vision-language-action (VLA) models such as pi0 generate robot actions by integrating a learned denoising velocity field, and have been reported to resist adversarial perturbations that readily fool autoregressive VLAs. We show that this robustness is largely illusory: it stems from prior attacks ignoring the multi-step denoising ODE. We introduce DRIFT (Denoising Redirection via Input perturbation of the Flow-matching Trajectory), a test-time universal adversarial patch placed on the robot's gripper that attacks the denoising velocity field of an off-the-shelf policy. Our central finding is counterintuitive: attacking only the first denoising step is both stronger and cheaper than attacking a wider window of steps, which we explain through a gradient conflict unique to input-space optimization and which is exactly opposite to the training-time backdoor regime. On pi0 and pi0.5 across four LIBERO suites, DRIFT breaks essentially all originally-solvable tasks with a small single patch, far exceeding action- and embedding-space attack baselines.