mlsanityData to 7 Oct 2026

Beyond the Patch: Exploring Vulnerabilities of Visuomotor Policies via Viewpoint-Consistent 3D Adversarial Object

arXiv 2603.04913 ↗First posted 5 Mar 2026cs.RO cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Neural network-based visuomotor policies enable robots to perform manipulation tasks but remain susceptible to perceptual attacks. For example, conventional 2D adversarial patches are effective under fixed-camera setups, where appearance is relatively consistent; however, their efficacy often diminishes under dynamic viewpoints from moving cameras, such as wrist-mounted setups, due to perspective distortions. To proactively investigate potential vulnerabilities beyond 2D patches, this work proposes a viewpoint-consistent adversarial texture optimization method for 3D objects through differentiable rendering. As optimization strategies, we employ Expectation over Transformation (EOT) with a Coarse-to-Fine (C2F) curriculum, exploiting distance-dependent frequency characteristics to induce textures effective across varying camera-object distances. We further integrate saliency-guided perturbations to redirect policy attention and design a targeted loss that persistently drives robots toward adversarial objects. Our comprehensive experiments show that the proposed method is effective under various environmental conditions, while confirming its black-box transferability and real-world applicability.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 35 references that are in the map. First posted on arXiv 5 Mar 2026.

Builds on

26 of 35 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2013Intriguing properties of neural networkscited by 5,114
  7. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  8. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  9. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  10. 2017Adversarial Patchcited by 514
  11. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 373
  12. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 216
14 more
  1. 2024Data Scaling Laws in Imitation Learning for Robotic Manipulationcited by 144
  2. 2017Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-To-End Learning from Demonstrationcited by 100
  3. 2021FCA: Learning a 3D Full-coverage Vehicle Camouflage for Multi-view Physical Adversarial Attackcited by 74
  4. 2017Adversarial Attacks Beyond the Image Spacecited by 67
  5. 2024OKAMI: Teaching Humanoid Robots Manipulation Skills through Single Video Imitationcited by 62
  6. 2022DTA: Physical Camouflage Attacks using Differentiable Transformation Networkcited by 51
  7. 2022Improving the Transferability of Targeted Adversarial Examples through Object-Based Diverse Inputcited by 49
  8. 2023Towards Transferable Targeted 3D Adversarial Attack in the Physical Worldcited by 25
  9. 2024LEGATO: Cross-Embodiment Imitation Using a Grasping Toolcited by 18
  10. 2024Diffusion Policy Attacker: Crafting Adversarial Attacks for Diffusion-based Policiescited by 17
  11. 2023Dynamic Adversarial Attacks on Autonomous Driving Systemscited by 11
  12. 2023Adv3D: Generating 3D Adversarial Examples for 3D Object Detection in Driving Scenarios with NeRFcited by 10
  13. 2022Synthesizing Adversarial Visual Scenarios for Model-Based Robotic Controlcited by 4
  14. 2023Generating Transferable Adversarial Simulation Scenarios for Self-Driving via Neural Renderingcited by 3

Built on by

Not cited yet.