mlsanityData to 7 Oct 2026

ASID: Active Exploration for System Identification in Robotic Manipulation

arXiv 2404.12308 ↗First posted 18 Apr 2024cs.RO cs.LG cs.SYcited by 27

Not on the map. Why?

Abstract (arXiv)

Model-free control strategies such as reinforcement learning have shown the ability to learn control strategies without requiring an accurate model or simulator of the world. While this is appealing due to the lack of modeling requirements, such methods can be sample inefficient, making them impractical in many real-world domains. On the other hand, model-based control techniques leveraging accurate simulators can circumvent these challenges and use a large amount of cheap simulation data to learn controllers that can effectively transfer to the real world. The challenge with such model-based techniques is the requirement for an extremely accurate simulation, requiring both the specification of appropriate simulation assets and physical parameters. This requires considerable human effort to design for every environment being considered. In this work, we propose a learning system that can leverage a small amount of real-world data to autonomously refine a simulation model and then plan an accurate control strategy that can be deployed in the real world. Our approach critically relies on utilizing an initial (possibly inaccurate) simulator to design effective exploration policies that, when deployed in the real world, collect high-quality data. We demonstrate the efficacy of this paradigm in identifying articulation, mass, and other physical parameters in several challenging robotic manipulation tasks, and illustrate that only a small amount of real-world data can allow for effective sim-to-real transfer. Project website at https://weirdlabuw.github.io/asid

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 49 of its 84 references that are in the map. First posted on arXiv 18 Apr 2024.

Builds on

49 of 84 references.

References · 49

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  4. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  5. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  6. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  7. 2020Mastering Atari with Discrete World Modelscited by 990
  8. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  9. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  10. 2018Learning Dexterous In-Hand Manipulationcited by 643
  11. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
  12. 2019Learning agile and dynamic motor skills for legged robotscited by 402
37 more
  1. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 395
  2. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 288
  3. 2019Self-Supervised Exploration via Disagreementcited by 263
  4. 2023Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuningcited by 230
  5. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 227
  6. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 203
  7. 2021Where2Act: From Pixels to Actions for Articulated 3D Objectscited by 179
  8. 2019Active Domain Randomizationcited by 161
  9. 2022Ditto: Building Digital Twins of Articulated Objects from Interactioncited by 152
  10. 2022DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Realitycited by 136
  11. 2024Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulationcited by 135
  12. 2018Model-Based Active Explorationcited by 105
  13. 2022In-Hand Object Rotation via Rapid Motor Adaptationcited by 102
  14. 2018Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Costcited by 79
  15. 2019BayesSim: adaptive domain randomization via probabilistic inference for robotics simulatorscited by 78
  16. 2020Active Learning for Nonlinear System Identification with Guaranteescited by 78
  17. 2021Legged Robots that Keep on Learning: Fine-Tuning Locomotion Policies in the Real Worldcited by 64
  18. 2019DensePhysNet: Learning Dense Physical Object Representations via Multi-step Dynamic Interactionscited by 61
  19. 2021AdaAfford: Learning to Adapt Manipulation Affordance for 3D Articulated Objects via Few-shot Interactionscited by 56
  20. 2023IndustReal: Transferring Contact-Rich Assembly Tasks from Simulation to Realitycited by 44
  21. 2023Ditto in the House: Building Articulation Models of Indoor Scenes through Interactive Perceptioncited by 40
  22. 2021Adaptive-Control-Oriented Meta-Learning for Nonlinear Systemscited by 38
  23. 2020Active Learning for Identification of Linear Dynamical Systemscited by 33
  24. 2019Assessing Transferability from Simulation to Reality for Reinforcement Learningcited by 29
  25. 2023AR2-D2:Training a Robot Without a Robotcited by 26
  26. 2023What Went Wrong? Closing the Sim-to-Real Gap via Differentiable Causal Discoverycited by 22
  27. 2020Reinforced Grounded Action Transformation for Sim-to-Real Transfercited by 17
  28. 2023AdaptSim: Task-Driven Simulation Adaptation for Sim-to-Real Transfercited by 17
  29. 2023Optimal Exploration for Model-Based RL in Nonlinear Systemscited by 17
  30. 2023RePo: Resilient Model-Based Reinforcement Learning by Regularizing Posterior Predictabilitycited by 17
  31. 2021Task-Optimal Exploration in Linear Dynamical Systemscited by 15
  32. 2024Offline Multi-task Transfer RL with Representational Penalizationcited by 13
  33. 2020Learning Active Task-Oriented Exploration Policies for Bridging the Sim-to-Real Gapcited by 11
  34. 2023Sim2Real2: Actively Building Explicit Physics Model for Precise Articulated Object Manipulationcited by 10
  35. 2021Adaptive Robust Model Predictive Control with Matched and Unmatched Uncertaintycited by 7
  36. 2020Scalable Identification of Partially Observed Systems with Certainty-Equivalent EMcited by 4
  37. 2022Dimensionality Reduction and Prioritized Exploration for Policy Searchcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 10; 2026: 12.

27 papers.

Sort:

By area:

  1. 2024Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imaginationcited by 45
  2. 2025Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulationcited by 42
  3. 2025Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoidscited by 40
  4. 2025PIN-WM: Learning Physics-INformed World Models for Non-Prehensile Manipulationcited by 22
  5. 2025The Reality Gap in Robotics: Challenges, Solutions, and Best Practicescited by 22
  6. 2024Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RLcited by 17
  7. 2025Rapidly Adapting Policies to the Real World via Simulation-Guided Fine-Tuningcited by 15
  8. 2024Dynamics as Prompts: In-Context Learning for Sim-to-Real System Identificationscited by 10
  9. 2024Robot Learning with Super-Linear Scalingcited by 10
  10. 2025OXE-AugE: A Large-Scale Robot Augmentation of OXE for Scaling Cross-Embodiment Policy Learningcited by 10
  11. 2026ContactGaussian-WM: Learning Physics-Grounded World Model from Videoscited by 7
  12. 2026Towards Exploratory and Focused Manipulation with Bimanual Active Perception: A New Problem, Benchmark and Strategycited by 4
15 more
  1. 2025LodeStar: Long-horizon Dexterity via Synthetic Data Augmentation from Human Demonstrationscited by 3
  2. 2024Incremental Few-Shot Adaptation for Non-Prehensile Object Manipulation using Parallelizable Physics Simulatorscited by 1
  3. 2025Neural Fidelity Calibration for Informative Sim-to-Real Adaptationcited by 1
  4. 2025ASBI: Leveraging Informative Real-World Data for Active Black-Box Simulator Tuningcited by 0
  5. 2025Posterior-driven Heuristic Support Adaptation in a Probabilistic Treatment of Real2Sim2Real for Vision-Driven Deformable Linear Object Manipulationcited by 0
  6. 2026Towards Batch-to-Streaming Deep Reinforcement Learning for Continuous Controlcited by 0
  7. 2026Learning What Matters: Adaptive Information-Theoretic Objectives for Robot Explorationcited by 0
  8. 2026TAM: Torque Adaptation Module for Robust Motion Transfer in Manipulationcited by 0
  9. 2026IDEA: Insensitive to Dynamics Mismatch via Effect Alignment for Sim-to-Real Transfer in Multi-Agent Controlcited by 0
  10. 2026Trajectory Design and Budgeted Querying for Digital Twin Calibrationcited by 0
  11. 2026Diagnose, Recover, Certify: Task Readiness under Hidden Dynamics Changescited by 0
  12. 2026When Does Test-Time Physical Diagnosis Pay? A Frozen Policy Buys Evidence It Never Readscited by 0
  13. 2026Test-Time Spatial Reasoning for Robot Manipulation Using Generative Real-to-Simcited by 0
  14. 2026ZeroBot: Learning from Scratch in Minutes with Generative Real2Simcited by 0
  15. 2026EMPIRIC: Experiment-Driven Learning of Residual World Models for Robot Planningcited by 0