TRAP: Tail-aware Ranking Attack for World-Model Planning
On the map: World models → World model security and GUI agents
also in 1 more
Abstract (arXiv)
World models enable long-horizon planning by internally generating and evaluating imagined trajectories, making them a promising foundation for generalist agents. However, this imagination-driven decision process also introduces new security risks. Existing backdoor attacks typically aim to manipulate local features, one-step predictions, or instantaneous policy outputs. While such objectives may suffice for weaker reactive models, they are often ineffective against world models, where the learned dynamics prior and planning process can absorb or wash out the effects of shallow perturbations. More importantly, we find that world models exhibit a distinct backdoor vulnerability rooted in the long-tailed ranking structure of imagined trajectories, where disrupting the ordering of a few decision-critical trajectories can systematically hijack planning. To exploit this vulnerability, we propose TRAP, a backdoor attack framework for world models that targets imagined trajectory ranking. TRAP combines a tail-aware ranking loss to focus optimization on decision-critical trajectories with dual gating mechanisms that stabilize optimization and regulate when and where the attack penalty is applied. Under trigger conditions, TRAP alters the relative ranking of imagined trajectories to redirect planning outcomes, while largely maintaining the normal ranking structure on clean inputs. Experiments on DreamerV3 and TD-MPC2 across diverse tasks show that TRAP consistently induces sustained behavioral deviations and significant performance degradation, highlighting the need for dedicated security evaluation of world-model-based agents.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in World model security and GUI agents, one of 11 directions in World models. Builds on 29 of its 39 references that are in the map. First posted on arXiv 3 May 2026.
Builds on
29 of 39 references.
References · 29
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
- 2018World Modelscited by 1,444
- 2023Mastering Diverse Domains through World Modelscited by 1,193
- 2020Mastering Atari with Discrete World Modelscited by 990
- 2018DeepMind Control Suitecited by 935
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
- 2017Countering Adversarial Images using Input Transformationscited by 517
- 2017Adversarial Patchcited by 514
- 2017Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networkscited by 484
- 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
- 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 204
- 2021Benchmarking the Spectrum of Agent Capabilitiescited by 193
17 more
- 2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 163
- 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
- 2018LaVAN: Localized and Visible Adversarial Noisecited by 112
- 2023STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learningcited by 88
- 2024WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokenscited by 87
- 2025Pre-Trained Video Generative Models as World Simulatorscited by 43
- 2017Synthesizing Robust Adversarial Examplescited by 39
- 2025RLVR-World: Training World Models with Reinforcement Learningcited by 38
- 2025BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimizationcited by 37
- 2023BadRL: Sparse Targeted Backdoor Attack Against Reinforcement Learningcited by 29
- 2022BAFFLE: Hiding Backdoors in Offline Reinforcement Learning Datasetscited by 24
- 2024RAT: Adversarial Attacks on Deep Reinforcement Agents for Targeted Behaviorscited by 16
- 2024Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretrainingcited by 13
- 2024Open-World Reinforcement Learning over Long Short-Term Imaginationcited by 13
- 2026State Backdoor: Towards Stealthy Real-world Poisoning Attack on Vision-Language-Action Model in State Spacecited by 9
- 2024Operator World Models for Reinforcement Learningcited by 7
- 2026Inject Once Survive Later: Backdooring Vision-Language-Action Models to Persist Through Downstream Fine-tuningcited by 7
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026WMAttack: Automated Attack Search for Adversarial Evaluation of World-Model Agentscited by 6
- 2026Security of World-Model-Based Embodied AI: A Lifecycle of Threats, Defenses, and Evaluationcited by 2
- 2026Where World Models Break: Natural-Input Failure Discoverycited by 0
- 2026TrojanWorld: Backdooring World-Model Agents via Imagination Steeringcited by 0
- 2026TrojanWorld: Backdooring World-Model Agents via Imagination Steering7 Sep 2026
- 2026Where World Models Break: Natural-Input Failure Discovery23 Aug 2026
- 2026Security of World-Model-Based Embodied AI: A Lifecycle of Threats, Defenses, and Evaluation30 Jul 2026
- 2026WMAttack: Automated Attack Search for Adversarial Evaluation of World-Model Agents22 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026When the World Lies: Backdoor Attacks on Latent World Models for Downstream Control
- 2026Safety, Security, and Cognitive Risks in World Models
- 2026Attacking the Trusted Imagination: Oracle-Level Integrity Attacks on Imagine-then-Act World Models
- 2026The Objective Is the Bottleneck: Latent World Models Encode What Their Planners Cannot Use
- 2026What Must a World Model Distinguish for Planning?
- 2026Aim Short to Reach Far: Your Frozen World Model Can Plan Better Than You Think
- 2026Dual-Frontier: When Can an Agent Trust Its World Model?
- 2026Stealthy World Model Manipulation via Data Poisoning