mlsanityData to 7 Oct 2026

SIME: Enhancing Policy Self-Improvement with Modal-level Exploration

arXiv 2505.01396 ↗First posted 2 May 2025cs.RO cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Self-improvement requires robotic systems to initially learn from human-provided data and then gradually enhance their capabilities through interaction with the environment. This is similar to how humans improve their skills through continuous practice. However, achieving effective self-improvement is challenging, primarily because robots tend to repeat their existing abilities during interactions, often failing to generate new, valuable data for learning. In this paper, we identify the key to successful self-improvement: modal-level exploration and data selection. By incorporating a modal-level exploration mechanism during policy execution, the robot can produce more diverse and multi-modal interactions. At the same time, we select the most valuable trials and high-quality segments from these interactions for learning. We successfully demonstrate effective robot self-improvement on both simulation benchmarks and real-world experiments. The capability for self-improvement will enable us to develop more robust and high-success-rate robotic control strategies at a lower cost. Our code and experiment scripts are available at https://ericjin2002.github.io/SIME/

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 33 of its 39 references that are in the map. First posted on arXiv 2 May 2025.

Builds on

33 of 39 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2023Segment Anythingcited by 8,158
  5. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  6. 2015Continuous control with deep reinforcement learningcited by 3,279
  7. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  8. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  9. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  10. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  11. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  12. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
21 more
  1. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  2. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  3. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  4. 2021Implicit Behavioral Cloningcited by 442
  5. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 373
  6. 2022Behavior Transformers: Cloning k modes with one stonecited by 299
  7. 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 173
  8. 2018Residual Policy Learningcited by 146
  9. 2023CADS: Unleashing the Diversity of Diffusion Models through Condition-Annealed Samplingcited by 124
  10. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learningcited by 113
  11. 2023RoboCat: A Self-Improving Generalist Agent for Robotic Manipulationcited by 108
  12. 2022Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deploymentcited by 101
  13. 2023Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrationscited by 74
  14. 2024Policy Decorator: Model-Agnostic Online Refinement for Large Policy Modelcited by 60
  15. 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 50
  16. 2024RISE: 3D Perception Makes Real-World Robot Imitation Simple and Effectivecited by 41
  17. 2022SAM-RL: Sensing-Aware Model-Based Reinforcement Learning via Differentiable Physics-Based Simulation and Renderingcited by 22
  18. 2024So You Think You Can Scale Up Autonomous Robot Data Collection?cited by 17
  19. 2025AirExo-2: Scaling up Generalizable Robotic Imitation Learning with Low-Cost Exoskeletonscited by 17
  20. 2024Human-Agent Joint Learning for Efficient Robot Manipulation Skill Acquisitioncited by 10
  21. 2024Towards Effective Utilization of Mixed-Quality Demonstrations in Robotic Manipulation via Segment-Level Selection and Optimizationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 3.

6 papers.

  1. 2025Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfercited by 13
  2. 2025SOE: Sample-Efficient Robot Policy Self-Improvement via On-Manifold Explorationcited by 5
  3. 2025RESample: A Robust Data Augmentation Framework via Exploratory Sampling for Robotic Manipulationcited by 5
  4. 2026WorldSample: Closed-loop Real-robot RL with World Modellingcited by 2
  5. 2026ExToken: Structured Exploration for Efficient Vision-Language-Action Reinforcement Fine-tuningcited by 2
  6. 2026Guided Discovery of New Behaviors using Diffusion Policiescited by 0