mlsanityData to 7 Oct 2026

ILBiT: Imitation Learning for Robot Using Position and Torque Information based on Bilateral Control with Transformer

arXiv 2401.16653 ↗First posted 30 Jan 2024cs.ROcited by 10

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Autonomous manipulation in robot arms is a complex and evolving field of study in robotics. This paper introduces an innovative approach to this challenge by focusing on imitation learning (IL). Unlike traditional imitation methods, our approach uses IL based on bilateral control, allowing for more precise and adaptable robot movements. The conventional IL based on bilateral control method have relied on Long Short-Term Memory (LSTM) networks. In this paper, we present the IL for robot using position and torque information based on Bilateral control with Transformer (ILBiT). This proposed method employs the Transformer model, known for its robust performance in handling diverse datasets and its capability to surpass LSTM's limitations, especially in tasks requiring detailed force adjustments. A standout feature of ILBiT is its high-frequency operation at 100 Hz, which significantly improves the system's adaptability and response to varying environments and objects of different hardness levels. The effectiveness of the Transformer-based ILBiT method can be seen through comprehensive real-world experiments.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 24 of its 51 references that are in the map. First posted on arXiv 30 Jan 2024.

Builds on

24 of 51 references.

Closest ancestors · 11

Same direction, newest first

  1. 2023Soft and Rigid Object Grasping With Cross-Structure Hand Using Bilateral Control-Based Imitation Learning16 Nov
  2. 2022Imitation Learning for Nonprehensile Manipulation through Self-Supervised Learning Considering Motion Speed21 Jun
  3. 2022Demonstrate Once, Imitate Immediately (DOME): Learning Visual Servoing for One-Shot Imitation Learning6 Apr
  4. 2022An Independently Learnable Hierarchical Model for Bilateral Control-Based Imitation Learning Applications16 Mar
  5. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning4 Feb
  6. 2021Transformer-based deep imitation learning for dual-arm robot manipulation1 Aug
  7. 2021Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstration13 May
  8. 2021Imitation Learning for Variable Speed Contact Motion for Operation up to Control Bandwidth20 Feb
  9. 2020Motion Generation Using Bilateral Control-Based Imitation Learning with Autoregressive Learning12 Nov
  10. 2019Learning One-Shot Imitation from Humans without Humans4 Nov
  11. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation7 Nov

Other references · 13

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  4. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  5. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 294
  6. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  7. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  8. 2021Learning by Watching: Physical Imitation of Manipulation Skills from Human Videoscited by 87
  9. 2020Learning Cross-Domain Correspondence for Control with Dynamics Cycle-Consistencycited by 43
  10. 2018Imitation Learning for Object Manipulation Based on Position/Force Information Using Bilateral Controlcited by 21
  11. 2019Time Series Motion Generation Considering Long Short-Term Motioncited by 10
  12. 2022Geometric Reinforcement Learning For Robotic Manipulationcited by 5
1 more
  1. 2023A Framework for Few-Shot Policy Transfer through Observation Mapping and Behavior Cloningcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 2; 2026: 4.

10 papers.

  1. 2024Bi-ACT: Bilateral Control-Based Imitation Learning via Action Chunking with Transformercited by 27
  2. 2024Mamba as a motion encoder for robotic imitation learningcited by 7
  3. 2024DABI: Evaluation of Data Augmentation Methods Using Downsampling in Bilateral Control-Based Imitation Learning with Imagescited by 7
  4. 2025ELMUR: External Layer Memory with Update/Rewrite for Long-Horizon RL Problemscited by 6
  5. 2025Society of Mind Meets Real-Time Strategy: A Hierarchical Multi-Agent Framework for Strategic Reasoningcited by 3
  6. 2024Variable-Speed Teaching-Playback as Real-World Data Augmentation for Imitation Learningcited by 2
  7. 2026Bi-HIL: Bilateral Control-Based Multimodal Hierarchical Imitation Learning via Subtask-Level Progress Rate and Keyframe Memory for Long-Horizon Contact-Rich Robotic Manipulationcited by 2
  8. 2026Force Generative Imitation Learning: Bridging Position Trajectory and Force Commands through Control Techniquecited by 0
  9. 2026NeuralActuator: Neural Actuation Modeling for Robot Dynamics and External Force Perceptioncited by 0
  10. 2026Learning human joint torques from pixelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024ALPHA-α and Bi-ACT Are All You Need: Importance of Position and Force Information/Control for Imitation Learning of Unimanual and Bimanual Robotic Manipulation with Low-Cost System
  2. 2025Learning Bimanual Manipulation via Action Chunking and Inter-Arm Coordination with Transformers
  3. 2025Interactive Imitation Learning for Dexterous Robotic Manipulation: Challenges and Perspectives -- A Survey
  4. 2025Robotic Manipulation via Imitation Learning: Taxonomy, Evolution, Benchmark, and Challenges
  5. 2025Dexterous Manipulation through Imitation Learning: A Survey
  6. 2025X-IL: Exploring the Design Space of Imitation Learning Policies
  7. 2025RoboManipBaselines: A Unified Framework for Imitation Learning in Robotic Manipulation across Real and Simulation Environments
  8. 2024A Comparison of Imitation Learning Algorithms for Bimanual Manipulation