# One Hand Watches The Other: Dynamic Multi-Agent Cooperation for Sample-Efficient Bimanual Manipulation in Dynamic Environments

arXiv: https://arxiv.org/abs/2607.22119 · HTML: https://mlsanity.com/paper/2607.22119/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 30 references that are in the map.
- First posted on arXiv 24 Jul 2026.

## Builds on (most cited first, 21 of 21)

- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [RLBench: The Robot Learning Benchmark & Learning Environment](https://mlsanity.com/paper/1909.12271/) (2019)
- [FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects](https://mlsanity.com/paper/2312.08344/) (2023)
- [THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation](https://mlsanity.com/paper/2402.08191/) (2024)
- [VLMbench: A Compositional Benchmark for Vision-and-Language Manipulation](https://mlsanity.com/paper/2206.08522/) (2022)
- [PerAct2: Benchmarking and Learning for Robotic Bimanual Manipulation Tasks](https://mlsanity.com/paper/2407.00278/) (2024)
- [Autoregressive Action Sequence Learning for Robotic Manipulation](https://mlsanity.com/paper/2410.03132/) (2024)
- [InterACT: Inter-dependency Aware Action Chunking with Hierarchical Attention Transformers for Bimanual Manipulation](https://mlsanity.com/paper/2409.07914/) (2024)
- [Towards Generalizable Robotic Manipulation in Dynamic Environments](https://mlsanity.com/paper/2603.15620/) (2026)
- [DualAfford: Learning Collaborative Visual Affordance for Dual-gripper Manipulation](https://mlsanity.com/paper/2207.01971/) (2022)
- [AnyBimanual: Transferring Unimanual Policy for General Bimanual Manipulation](https://mlsanity.com/paper/2412.06779/) (2024)
- [The Treachery of Images: Bayesian Scene Keypoints for Deep Policy Learning in Robotic Manipulation](https://mlsanity.com/paper/2305.04718/) (2023)
- [Rethinking Bimanual Robotic Manipulation: Learning with Decoupled Interaction Framework](https://mlsanity.com/paper/2503.09186/) (2025)
- [The Art of Imitation: Learning Long-Horizon Manipulation Tasks from Few Demonstrations](https://mlsanity.com/paper/2407.13432/) (2024)
- [Learning and Sequencing of Object-Centric Manipulation Skills for Industrial Tasks](https://mlsanity.com/paper/2008.10471/) (2020)
- [RoboStream: Weaving Spatio-Temporal Reasoning with Memory in Vision-Language Models for Robotics](https://mlsanity.com/paper/2603.12939/) (2026)
- [Exploiting Information Theory for Intuitive Robot Programming of Manual Activities](https://mlsanity.com/paper/2410.23963/) (2024)
- [The Unreasonable Effectiveness of Discrete-Time Gaussian Process Mixtures for Robot Policy Learning](https://mlsanity.com/paper/2505.03296/) (2025)
- [MSG: Multi-Stream Generative Policies for Sample-Efficient Robotic Manipulation](https://mlsanity.com/paper/2509.24956/) (2025)
- [SparTa: Sparse Graphical Task Models from a Handful of Demonstrations](https://mlsanity.com/paper/2602.16911/) (2026)

## Abstract (arXiv)

Multi-stream robot manipulation policies achieve unparalleled sample efficiency and generalization by modeling actions relative to environmental reference frames. However, existing approaches typically assume these frames to be strictly exogenous. This causal assumption collapses in dynamic settings, such as when a single robot arm manipulates a moving object or when two arms coordinate, where each arm effectively becomes part of the dynamic environment of the other. We propose DynaMAC, a lightweight, policy-agnostic framework that resolves this causal limitation while preserving the sample efficiency, computational speed, and flexibility of multi-stream policies, DynaMAC treats the opposite arm as a dynamic task parameter, thereby providing a unified formulation for dynamic manipulation and bimanual coordination without requiring an explicit leader-follower relationship. To rigorously evaluate these capabilities, we introduce DynaBench, a novel benchmark for robot manipulation in dynamic environments. Across both dynamic environments and bimanual manipulation tasks, DynaMAC outperforms leading probabilistic and generative baselines by over 35 percentage points while requiring 20 times fewer samples. Crucially, DynaMAC generalizes zero-shot from static demonstrations to dynamic environments, substantially simplifying data collection and establishing an elegant bridge toward human-robot collaboration.