mlsanityData to 7 Oct 2026

State-Only Imitation Learning for Dexterous Manipulation

arXiv 2004.04650 ↗First posted 7 Apr 2020cs.RO cs.LG stat.MLcited by 90

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Modern model-free reinforcement learning methods have recently demonstrated impressive results on a number of problems. However, complex domains like dexterous manipulation remain a challenge due to the high sample complexity. To address this, current approaches employ expert demonstrations in the form of state-action pairs, which are difficult to obtain for real-world settings such as learning from videos. In this paper, we move toward a more realistic setting and explore state-only imitation learning. To tackle this setting, we train an inverse dynamics model and use it to predict actions for state-only demonstrations. The inverse dynamics model and the policy are trained jointly. Our method performs on par with state-action approaches and considerably outperforms RL alone. By not relying on expert actions, we are able to learn from demonstrations with different dynamics, morphologies, and objects. Videos available at https://people.eecs.berkeley.edu/~ilija/soil .

Key facts. Cited by 90 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 36 of its 60 references that are in the map. First posted on arXiv 7 Apr 2020.

Builds on

36 of 60 references.

Closest ancestors · 2

Same direction, newest first

  1. 2018Zero-Shot Visual Imitation23 Apr
  2. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skills26 Feb

Other references · 34

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2016Generative Adversarial Imitation Learningcited by 1,576
  4. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  5. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  6. 2016End to End Learning for Self-Driving Carscited by 1,000
  7. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  8. 2019Solving Rubik's Cube with a Robot Handcited by 699
  9. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  10. 2018Learning Dexterous In-Hand Manipulationcited by 643
  11. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  12. 2018Behavioral Cloning from Observationcited by 482
22 more
  1. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  2. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 361
  3. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
  4. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 288
  5. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 227
  6. 2019Deep Dynamics Models for Learning Dexterous Manipulationcited by 200
  7. 2018Generative Adversarial Imitation from Observationcited by 183
  8. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  9. 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 153
  10. 2016Learning to Poke by Poking: Experiential Learning of Intuitive Physicscited by 152
  11. 2019DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm Systemcited by 105
  12. 2018Imitating Latent Policies from Observationcited by 101
  13. 2018Playing hard exploration games by watching YouTubecited by 98
  14. 2017Combining Self-Supervised Learning and Imitation for Vision-Based Rope Manipulationcited by 90
  15. 2019Provably Efficient Imitation Learning from Observation Alonecited by 78
  16. 2018Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitationcited by 76
  17. 2019State Alignment-based Imitation Learningcited by 73
  18. 2016Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Modelcited by 53
  19. 2016The Curious Robot: Learning Visual Representations via Physical Interactionscited by 37
  20. 2019RIDM: Reinforced Inverse Dynamics Modeling for Learning from a Single Observed Demonstrationcited by 19
  21. 2019Hybrid Reinforcement Learning with Expert State Sequencescited by 14
  22. 2019Learning Navigation Subroutines from Egocentric Videoscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 26; 2023: 20; 2024: 18; 2025: 12; 2026: 4.

90 papers.

Sort:

By area:

  1. 2022Masked Visual Pre-training for Motor Controlcited by 246
  2. 2021DexMV: Imitation Learning for Dexterous Manipulation from Human Videoscited by 194
  3. 2021Hand-Object Contact Consistency Reasoning for Human Grasps Generationcited by 160
  4. 2021A System for General In-Hand Object Re-Orientationcited by 155
  5. 2022OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interactioncited by 129
  6. 2022In-Hand Object Rotation via Rapid Motor Adaptationcited by 102
  7. 2021XIRL: Cross-embodiment Inverse Reinforcement Learningcited by 101
  8. 2023UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learningcited by 101
  9. 2025DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Graspingcited by 87
  10. 2022DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Videocited by 83
  11. 2022Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtubecited by 79
  12. 2022From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperationcited by 79
38 more
  1. 2022DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulationcited by 78
  2. 2022Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulationcited by 75
  3. 2023Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrationscited by 74
  4. 2022Watch and Match: Supercharging Imitation with Regularized Optimal Transportcited by 64
  5. 2023DexArt: Benchmarking Generalizable Dexterous Manipulation with Articulated Objectscited by 64
  6. 2022MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrationscited by 59
  7. 2023Diffusion Reward: Learning Rewards via Conditional Video Diffusioncited by 56
  8. 2022GenDexGrasp: Generalizable Dexterous Graspingcited by 54
  9. 2022Holo-Dex: Teaching Dexterity with Immersive Mixed Realitycited by 52
  10. 2021Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learningcited by 49
  11. 2022Learning Generalizable Dexterous Manipulation from Human Grasp Affordancecited by 46
  12. 2022Graph Inverse Reinforcement Learning from Diverse Videoscited by 45
  13. 2022Versatile Offline Imitation from Observations and Examples via Regularized State-Occupancy Matchingcited by 43
  14. 2023Detecting Human-Object Contact in Imagescited by 41
  15. 2023Masked Trajectory Models for Prediction, Representation, and Controlcited by 40
  16. 2024ViViDex: Learning Vision-based Dexterous Manipulation from Human Videoscited by 33
  17. 2022Learning to Imitate Object Interactions from Internet Videoscited by 32
  18. 2023Hyper-Decision Transformer for Efficient Online Policy Adaptationcited by 32
  19. 2023RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learningcited by 29
  20. 2023LS-IQ: Implicit Reward Regularization for Inverse Reinforcement Learningcited by 28
  21. 2024Grounding Video Models to Actions through Goal Conditioned Explorationcited by 25
  22. 2023DexDeform: Dexterous Deformable Object Manipulation with Human Demonstrations and Differentiable Physicscited by 23
  23. 2023GraspGF: Learning Score-based Grasping Primitive for Human-assisting Dexterous Graspingcited by 23
  24. 2025SKIL: Semantic Keypoint Imitation Learning for Generalizable Data-efficient Manipulationcited by 21
  25. 2022DexTransfer: Real World Multi-fingered Dexterous Grasping with Minimal Human Demonstrationscited by 19
  26. 2023MAHALO: Unifying Offline Reinforcement Learning and Imitation Learning from Observationscited by 18
  27. 2024PhysReaction: Physically Plausible Real-Time Humanoid Reaction Synthesis via Forward Dynamics Guided 4D Imitationcited by 18
  28. 2021H2O: A Benchmark for Visual Human-human Object Handover Analysiscited by 16
  29. 2024A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agentscited by 16
  30. 2025DEXOP: A Device for Robotic Transfer of Dexterous Human Manipulationcited by 15
  31. 2022Learn what matters: cross-domain imitation learning with task-relevant embeddingscited by 14
  32. 2021Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Taskscited by 13
  33. 2025Responsive Noise-Relaying Diffusion Policy: Responsive and Efficient Visuomotor Controlcited by 13
  34. 2022REvolveR: Continuous Evolutionary Models for Robot-to-robot Policy Transfercited by 12
  35. 2024PROGRESSOR: A Perceptually Guided Reward Estimator with Self-Supervised Online Refinementcited by 11
  36. 2021Learning Feasibility to Imitate Demonstrators with Different Dynamicscited by 10
  37. 2023Imitation Learning from Observation with Automatic Discount Schedulingcited by 10
  38. 2024QuasiSim: Parameterized Quasi-Physical Simulators for Dexterous Manipulations Transfercited by 10

50 most cited of 90 All 90 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Dexterous Manipulation through Imitation Learning: A Survey
  2. 2020Model-Based Inverse Reinforcement Learning from Visual Demonstrations
  3. 2026Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards
  4. 2026DexPIE: Stable Dexterous Policy Improvement from Real-World Experience
  5. 2021Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstration
  6. 2023REBOOT: Reuse Data for Bootstrapping Efficient Real-World Dexterous Manipulation
  7. 2025DexForce: Extracting Force-informed Actions from Kinesthetic Demonstrations for Dexterous Manipulation
  8. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning