# Embodied AI with Two Arms: Zero-shot Learning, Safety and Modularity

arXiv: https://arxiv.org/abs/2404.03570 · HTML: https://mlsanity.com/paper/2404.03570/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Apr 2024.

## Built on by (most cited first, 19 of 19)

- [Generating Robot Constitutions & Benchmarks for Semantic Safety](https://mlsanity.com/paper/2503.08663/) (2025)
- [A Survey of Robotic Language Grounding: Tradeoffs between Symbols and Embeddings](https://mlsanity.com/paper/2405.13245/) (2024)
- [InterACT: Inter-dependency Aware Action Chunking with Hierarchical Attention Transformers for Bimanual Manipulation](https://mlsanity.com/paper/2409.07914/) (2024)
- [VoxAct-B: Voxel-Based Acting and Stabilizing Policy for Bimanual Manipulation](https://mlsanity.com/paper/2407.04152/) (2024)
- [AnyBimanual: Transferring Unimanual Policy for General Bimanual Manipulation](https://mlsanity.com/paper/2412.06779/) (2024)
- [Robot Collapse: Supply Chain Backdoor Attacks Against VLM-based Robotic Manipulation](https://mlsanity.com/paper/2411.11683/) (2024)
- [CREST: Cross-modal Resonance through Evidential Deep Learning for Enhanced Zero-Shot Learning](https://mlsanity.com/paper/2404.09640/) (2024)
- [Modeling the Real World with High-Density Visual Particle Dynamics](https://mlsanity.com/paper/2406.19800/) (2024)
- [DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robots](https://mlsanity.com/paper/2409.06613/) (2024)
- [Adaptive Masking Enhances Visual Grounding](https://mlsanity.com/paper/2410.03161/) (2024)
- [What Breaks Embodied AI Security:LLM Vulnerabilities, CPS Flaws,or Something Else?](https://mlsanity.com/paper/2602.17345/) (2026)
- [BiKC: Keypose-Conditioned Consistency Policy for Bimanual Robotic Manipulation](https://mlsanity.com/paper/2406.10093/) (2024)
- [Using large language models for embodied planning introduces systematic safety risks](https://mlsanity.com/paper/2604.18463/) (2026)
- [Contextual Safety Reasoning and Grounding for Open-World Robots](https://mlsanity.com/paper/2602.19983/) (2026)
- [ProVox: Personalization and Proactive Planning for Situated Human-Robot Collaboration](https://mlsanity.com/paper/2506.12248/) (2025)
- [From Code to Action: Hierarchical Learning of Diffusion-VLM Policies](https://mlsanity.com/paper/2509.24917/) (2025)
- [WINFlowNets: Warm-up Integrated Networks Training of Generative Flow Networks for Robotics and Machine Fault Adaptation](https://mlsanity.com/paper/2603.17301/) (2026)
- [Security and Privacy in Large-Model-Driven Embodied Agents: Attacks, Defenses, and Future Directions](https://mlsanity.com/paper/2609.27847/) (2026)
- [Controlling Collectives of AI Agents in Reasoning Space with Spatial Transformers](https://mlsanity.com/paper/2609.28247/) (2026)

## Abstract (arXiv)

We present an embodied AI system which receives open-ended natural language instructions from a human, and controls two arms to collaboratively accomplish potentially long-horizon tasks over a large workspace. Our system is modular: it deploys state of the art Large Language Models for task planning,Vision-Language models for semantic perception, and Point Cloud transformers for grasping. With semantic and physical safety in mind, these modules are interfaced with a real-time trajectory optimizer and a compliant tracking controller to enable human-robot proximity. We demonstrate performance for the following tasks: bi-arm sorting, bottle opening, and trash disposal tasks. These are done zero-shot where the models used have not been trained with any real world data from this bi-arm robot, scenes or workspace. Composing both learning- and non-learning-based components in a modular fashion with interpretable inputs and outputs allows the user to easily debug points of failures and fragilities. One may also in-place swap modules to improve the robustness of the overall platform, for instance with imitation-learned policies. Please see https://sites.google.com/corp/view/safe-robots .