# TAPDreamer: Transferable Adversarial Patches for World Action Models

arXiv: https://arxiv.org/abs/2610.06814 · HTML: https://mlsanity.com/paper/2610.06814/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Embodied and robot world models, one of 11 directions in World models.
- Builds on 38 of its 38 references that are in the map.
- First posted on arXiv 5 Oct 2026.

## Placed in

- World models / [Embodied and robot world models](https://mlsanity.com/wm/embodied-and-robot-world-models/)

## Builds on (most cited first, 30 of 38)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Improving Transferability of Adversarial Examples with Input Diversity](https://mlsanity.com/paper/1803.06978/) (2018)
- [Delving into Transferable Adversarial Examples and Black-box Attacks](https://mlsanity.com/paper/1611.02770/) (2016)
- [Learning Universal Policies via Text-Guided Video Generation](https://mlsanity.com/paper/2302.00111/) (2023)
- [RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation](https://mlsanity.com/paper/2506.18088/) (2025)
- [World Action Models are Zero-shot Policies](https://mlsanity.com/paper/2602.15922/) (2026)
- [Fast-WAM: Do World Action Models Need Test-time Future Imagination?](https://mlsanity.com/paper/2603.16666/) (2026)
- [Motus: A Unified Latent Action World Model](https://mlsanity.com/paper/2512.13030/) (2025)
- [Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets](https://mlsanity.com/paper/2504.02792/) (2025)
- [Adversarial Policies: Attacking Deep Reinforcement Learning](https://mlsanity.com/paper/1905.10615/) (2019)
- [RoboDreamer: Learning Compositional World Models for Robot Imagination](https://mlsanity.com/paper/2404.12377/) (2024)
- [Raising the Cost of Malicious AI-Powered Image Editing](https://mlsanity.com/paper/2302.06588/) (2023)
- [Feature Importance-aware Transferable Adversarial Attacks](https://mlsanity.com/paper/2107.14185/) (2021)
- [LaVAN: Localized and Visible Adversarial Noise](https://mlsanity.com/paper/1801.02608/) (2018)
- [Improving Adversarial Transferability via Neuron Attribution-Based Attacks](https://mlsanity.com/paper/2204.00008/) (2022)
- [Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Robotics](https://mlsanity.com/paper/2411.13587/) (2024)
- [Robust Physical-World Attacks on Deep Learning Models](https://mlsanity.com/paper/1707.08945/) (2017)
- [Adversarial camera stickers: A physical camera-based attack on deep learning systems](https://mlsanity.com/paper/1904.00759/) (2019)
- [Fast Feature Fool: A data independent approach to universal adversarial perturbations](https://mlsanity.com/paper/1707.05572/) (2017)
- [FDA: Feature Disruptive Attack](https://mlsanity.com/paper/1909.04385/) (2019)
- [Patch-Fool: Are Vision Transformers Always Robust Against Adversarial Perturbations?](https://mlsanity.com/paper/2203.08392/) (2022)
- [Boosting Adversarial Attacks with Momentum](https://mlsanity.com/paper/1710.06081/) (2017)
- [Model-agnostic Adversarial Attack and Defense for Vision-Language-Action Models](https://mlsanity.com/paper/2510.13237/) (2025)
- [FreezeVLA: Action-Freezing Attacks against Vision-Language-Action Models](https://mlsanity.com/paper/2509.19870/) (2025)
- [When Alignment Fails: Multimodal Adversarial Attacks on Vision-Language-Action Models](https://mlsanity.com/paper/2511.16203/) (2025)
- [When Robots Obey the Patch: Universal Transferable Patch Attacks on Vision-Language-Action Models](https://mlsanity.com/paper/2511.21192/) (2025)

## Abstract (arXiv)

World models learn to predict how their environment will evolve, making them an important foundation for general-purpose robotic control. Yet world action models depend on camera inputs whose manipulation can corrupt the visual representations used across tasks and action policies. Existing attacks on these models optimize against the victim's actions or predicted futures and therefore require access to target-model outputs. In this paper, we propose an attack, TAPDreamer, against world action models that instead uses a public encoder alone to construct a fixed local perturbation that transfers across tasks and action architectures. TAPDreamer requires no target-policy queries. Our key insight is that interactions between patch-induced changes in attention weights and value vectors broadcast a nearly identical representation shift far beyond the patch footprint, and this shift remains stable across task observations. Guided by this insight, TAPDreamer uses six frames from one source task to maximize the global L1 distance between clean and patched encoder representations. In closed-loop evaluation, one frozen patch per benchmark, covering about 6.5% of the input, reduces FastWAM's success rate from 97.7% to 0.0% across 40 LIBERO tasks and from 90.86% to 0.0% across 50 RoboTwin tasks; matched random patches retain 81.5% and 79.2% success. The same patches reduce success to 1.45% and 1.00% on two DreamWAM configurations and to 10.60% on Motus. These results show that protecting downstream action generation alone is insufficient: defenses for world action models must also secure shared visual encoders against persistent local perturbations.