# Geometric Red-Teaming for Robotic Manipulation

arXiv: https://arxiv.org/abs/2509.12379 · HTML: https://mlsanity.com/paper/2509.12379/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 15 Sep 2025.

## Builds on (most cited first, 24 of 24)

- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [RLBench: The Robot Learning Benchmark & Learning Environment](https://mlsanity.com/paper/1909.12271/) (2019)
- [SAPIEN: A SimulAted Part-based Interactive ENvironment](https://mlsanity.com/paper/2003.08515/) (2020)
- [robosuite: A Modular Simulation Framework and Benchmark for Robot Learning](https://mlsanity.com/paper/2009.12293/) (2020)
- [Certifying LLM Safety against Adversarial Prompting](https://mlsanity.com/paper/2309.02705/) (2023)
- [Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?](https://mlsanity.com/paper/2310.10012/) (2023)
- [Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes](https://mlsanity.com/paper/2103.14127/) (2021)
- [ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations](https://mlsanity.com/paper/2107.14483/) (2021)
- [AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation](https://mlsanity.com/paper/2410.00371/) (2024)
- [Curiosity-driven Red-teaming for Large Language Models](https://mlsanity.com/paper/2402.19464/) (2024)
- [Approximate Convex Decomposition for 3D Meshes with Collision-Aware Concavity and Tree Search](https://mlsanity.com/paper/2205.02961/) (2022)
- [EGAD! an Evolved Grasping Analysis Dataset for diversity and reproducibility in robotic manipulation](https://mlsanity.com/paper/2003.01314/) (2020)
- [IndustReal: Transferring Contact-Rich Assembly Tasks from Simulation to Reality](https://mlsanity.com/paper/2305.17110/) (2023)
- [DoorGym: A Scalable Door Opening Environment And Baseline Agent](https://mlsanity.com/paper/1908.01887/) (2019)
- [As-Plausible-As-Possible: Plausibility-Aware Mesh Deformation Using 2D Diffusion Priors](https://mlsanity.com/paper/2311.16739/) (2023)
- [Embodied Red Teaming for Auditing Robotic Foundation Models](https://mlsanity.com/paper/2411.18676/) (2024)
- [Evolving Diverse Red-team Language Models in Multi-round Multi-agent Games](https://mlsanity.com/paper/2310.00322/) (2023)
- [Predictive Red Teaming: Breaking Policies Without Breaking Robots](https://mlsanity.com/paper/2502.06575/) (2025)
- [Groot: Adversarial Testing for Generative Text-to-Image Models with Tree-based Semantic Transformation](https://mlsanity.com/paper/2402.12100/) (2024)
- [Solving Challenging Dexterous Manipulation Tasks With Trajectory Optimisation and Reinforcement Learning](https://mlsanity.com/paper/2009.05104/) (2020)
- [RoboMD: Uncovering Robot Vulnerabilities through Semantic Potential Fields](https://mlsanity.com/paper/2412.02818/) (2024)
- [Red-Teaming Segment Anything Model](https://mlsanity.com/paper/2404.02067/) (2024)

## Built on by (most cited first, 4 of 4)

- [What Breaks Embodied AI Security:LLM Vulnerabilities, CPS Flaws,or Something Else?](https://mlsanity.com/paper/2602.17345/) (2026)
- [Red-Teaming Vision-Language-Action Models via Quality Diversity Prompt Generation for Robust Robot Policies](https://mlsanity.com/paper/2603.12510/) (2026)
- [Uncovering Linguistic Fragility in Vision-Language-Action Models via Diversity-Aware Red Teaming](https://mlsanity.com/paper/2604.05595/) (2026)
- [ProbeAct: Probe-Guided Training-Free Failure Recovery in Vision-Language-Action Models](https://mlsanity.com/paper/2606.09740/) (2026)

## Abstract (arXiv)

Standard evaluation protocols in robotic manipulation typically assess policy performance over curated, in-distribution test sets, offering limited insight into how systems fail under plausible variation. We introduce Geometric Red-Teaming (GRT), a red-teaming framework that probes robustness through object-centric geometric perturbations, automatically generating CrashShapes -- structurally valid, user-constrained mesh deformations that trigger catastrophic failures in pre-trained manipulation policies. The method integrates a Jacobian field-based deformation model with a gradient-free, simulator-in-the-loop optimization strategy. Across insertion, articulation, and grasping tasks, GRT consistently discovers deformations that collapse policy performance, revealing brittle failure modes missed by static benchmarks. By combining task-level policy rollouts with constraint-aware shape exploration, we aim to build a general purpose framework for structured, object-centric robustness evaluation in robotic manipulation. We additionally show that fine-tuning on individual CrashShapes, a process we refer to as blue-teaming, improves task success by up to 60 percentage points on those shapes, while preserving performance on the original object, demonstrating the utility of red-teamed geometries for targeted policy refinement. Finally, we validate both red-teaming and blue-teaming results with a real robotic arm, observing that simulated CrashShapes reduce task success from 90% to as low as 22.5%, and that blue-teaming recovers performance to up to 90% on the corresponding real-world geometry -- closely matching simulation outcomes. Videos and code can be found on our project website: https://georedteam.github.io/ .