mlsanityData to 7 Oct 2026

Bi-ACT: Bilateral Control-Based Imitation Learning via Action Chunking with Transformer

arXiv 2401.17698 ↗First posted 31 Jan 2024cs.ROcited by 27

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Autonomous manipulation in robot arms is a complex and evolving field of study in robotics. This paper proposes work stands at the intersection of two innovative approaches in the field of robotics and machine learning. Inspired by the Action Chunking with Transformer (ACT) model, which employs joint location and image data to predict future movements, our work integrates principles of Bilateral Control-Based Imitation Learning to enhance robotic control. Our objective is to synergize these techniques, thereby creating a more robust and efficient control mechanism. In our approach, the data collected from the environment are images from the gripper and overhead cameras, along with the joint angles, angular velocities, and forces of the follower robot using bilateral control. The model is designed to predict the subsequent steps for the joint angles, angular velocities, and forces of the leader robot. This predictive capability is crucial for implementing effective bilateral control in the follower robot, allowing for more nuanced and responsive maneuvering.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 20 of its 32 references that are in the map. First posted on arXiv 31 Jan 2024.

Builds on

20 of 32 references.

Closest ancestors · 12

Same direction, newest first

  1. 2024ILBiT: Imitation Learning for Robot Using Position and Torque Information based on Bilateral Control with Transformer30 Jan
  2. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation4 Jan
  3. 2023Soft and Rigid Object Grasping With Cross-Structure Hand Using Bilateral Control-Based Imitation Learning16 Nov
  4. 2023GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators22 Sep
  5. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware23 Apr
  6. 2022Interactive Imitation Learning of Bimanual Movement Primitives28 Oct
  7. 2022Imitation Learning for Nonprehensile Manipulation through Self-Supervised Learning Considering Motion Speed21 Jun
  8. 2022An Independently Learnable Hierarchical Model for Bilateral Control-Based Imitation Learning Applications16 Mar
  9. 2021Imitation Learning for Variable Speed Contact Motion for Operation up to Control Bandwidth20 Feb
  10. 2020Learning Multi-Arm Manipulation Through Collaborative Teleoperation12 Dec
  11. 2020Grasping with Chopsticks: Combating Covariate Shift in Model-free Imitation Learning for Fine Manipulation13 Nov
  12. 2020Motion Generation Using Bilateral Control-Based Imitation Learning with Autoregressive Learning12 Nov

Other references · 8

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  4. 2017A survey of robot learning from demonstrations for Human-Robot Collaborationcited by 38
  5. 2022Causal Imitation Learning under Temporally Correlated Noisecited by 28
  6. 2018Imitation Learning for Object Manipulation Based on Position/Force Information Using Bilateral Controlcited by 21
  7. 2023One ACT Play: Single Demonstration Behavior Cloning with Action Chunking Transformerscited by 17
  8. 2019Time Series Motion Generation Considering Long Short-Term Motioncited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 12; 2026: 11.

27 papers.

Sort:

By area:

  1. 2025Reactive Diffusion Policy: Slow-Fast Visual-Tactile Policy Learning for Contact-Rich Manipulationcited by 103
  2. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  3. 2025RoboCopilot: Human-in-the-loop Interactive Imitation Learning for Robot Manipulationcited by 22
  4. 2024Learning Variable Compliance Control From a Few Demonstrations for Bimanual Robot with Haptic Feedback Teleoperation Systemcited by 13
  5. 2025Learning Bimanual Manipulation via Action Chunking and Inter-Arm Coordination with Transformerscited by 8
  6. 2024DABI: Evaluation of Data Augmentation Methods Using Downsampling in Bilateral Control-Based Imitation Learning with Imagescited by 7
  7. 2025Bi-LAT: Bilateral Control-Based Imitation Learning via Natural Language and Action Chunking with Transformerscited by 5
  8. 2025Bi-VLA: Bilateral Control-Based Imitation Learning via Vision-Language Fusion for Action Generationcited by 3
  9. 2026MA-VLA: Multi-Arm Vision-Language-Action Model for Collaboration and Compositional Generalizationcited by 3
  10. 2024Variable-Speed Teaching-Playback as Real-World Data Augmentation for Imitation Learningcited by 2
  11. 2025Design and Experimental Validation of Sensorless 4-Channel Bilateral Teleoperation for Low-Cost Manipulatorscited by 2
  12. 2025GauDP: Reinventing Multi-Agent Collaboration through Gaussian-Image Synergy in Diffusion Policiescited by 2
15 more
  1. 2026Bi-HIL: Bilateral Control-Based Multimodal Hierarchical Imitation Learning via Subtask-Level Progress Rate and Keyframe Memory for Long-Horizon Contact-Rich Robotic Manipulationcited by 2
  2. 2026YUBI: Yielding Universal Bidigital Interface for Bimanual Dexterous Manipulation at Scalecited by 2
  3. 2024Error-Feedback Model for Output Correction in Bilateral Control-Based Imitation Learningcited by 1
  4. 2025Improving Low-Cost Teleoperation: Augmenting GELLO with Forcecited by 1
  5. 2026RoboGene: Boosting VLA Pre-training via Diversity-Driven Agentic Framework for Real-World Task Generationcited by 1
  6. 2026FACTR 2: Learning External Force Sensing for Commodity Robot Arms Improves Policy Learningcited by 1
  7. 2026FWBC-VLA: Force-Aware Whole-Body Compensation for Contact-Rich Loco-Manipulationcited by 1
  8. 2025BEAC: Imitating Complex Exploration and Task-oriented Behaviors for Invisible Object Nonprehensile Manipulationcited by 0
  9. 2025A3RNN: Bi-directional Fusion of Bottom-up and Top-down Process for Developmental Visual Attention in Robotscited by 0
  10. 2025VRScout: Towards Real-Time, Autonomous Testing of Virtual Reality Gamescited by 0
  11. 2026Force Generative Imitation Learning: Bridging Position Trajectory and Force Commands through Control Techniquecited by 0
  12. 2026Active Stereo-Camera Outperforms Multi-Sensor Setup in ACT Imitation Learning for Humanoid Manipulationcited by 0
  13. 2026Bimanual Robot Manipulation via Multi-Agent In-Context Learningcited by 0
  14. 2026Continuous Manifold-Decomposed Impedance Retargeting for Contact-Rich Imitation Learningcited by 0
  15. 2026Bi-MoDe: Bilateral Control-based Imitation Learning via Modifier-Conditioned Decoding for Modulation of Execution Speed and Contact Intensitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024ALPHA-α and Bi-ACT Are All You Need: Importance of Position and Force Information/Control for Imitation Learning of Unimanual and Bimanual Robotic Manipulation with Low-Cost System
  2. 2024A Comparison of Imitation Learning Algorithms for Bimanual Manipulation
  3. 2024BiKC: Keypose-Conditioned Consistency Policy for Bimanual Robotic Manipulation
  4. 2026EquiBim: Learning Symmetry-Equivariant Policy for Bimanual Manipulation
  5. 2024InterACT: Inter-dependency Aware Action Chunking with Hierarchical Attention Transformers for Bimanual Manipulation
  6. 2021Bilateral Control-Based Imitation Learning for Velocity-Controlled Robot
  7. 2022A System for Imitation Learning of Contact-Rich Bimanual Manipulation Policies
  8. 2025LiPo: A Lightweight Post-optimization Framework for Smoothing Action Chunks Generated by Learned Policies