mlsanityData to 14 Oct 2026

Training Robots to Evaluate Robots: Example-Based Interactive Reward Functions for Policy Learning

arXiv 2212.08961 ↗First posted 17 Dec 2022cs.LG cs.ROcited by 1

Not on the map. Why?

Abstract (arXiv)

Physical interactions can often help reveal information that is not readily apparent. For example, we may tug at a table leg to evaluate whether it is built well, or turn a water bottle upside down to check that it is watertight. We propose to train robots to acquire such interactive behaviors automatically, for the purpose of evaluating the result of an attempted robotic skill execution. These evaluations in turn serve as "interactive reward functions" (IRFs) for training reinforcement learning policies to perform the target skill, such as screwing the table leg tightly. In addition, even after task policies are fully trained, IRFs can serve as verification mechanisms that improve online task execution. For any given task, our IRFs can be conveniently trained using only examples of successful outcomes, and no further specification is needed to train the task policy thereafter. In our evaluations on door locking and weighted block stacking in simulation, and screw tightening on a real robot, IRFs enable large performance improvements, even outperforming baselines with access to demonstrations or carefully engineered rewards. Project website: https://sites.google.com/view/lirf-corl-2022/

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 34 of its 49 references that are in the map. First posted on arXiv 17 Dec 2022.

Builds on

34 of 49 references.

References · 34

  1. 2022Generative Adversarial Networkscited by 5,627
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2017Deep reinforcement learning from human preferencescited by 3,912
  4. 2016OpenAI Gymcited by 1,736
  5. 2016Generative Adversarial Imitation Learningcited by 1,597
  6. 2017Deep Setscited by 1,421
  7. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  8. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  9. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
  10. 2017Robust Adversarial Reinforcement Learningcited by 451
  11. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 298
  12. 2018Generative Adversarial Imitation from Observationcited by 184
22 more
  1. 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 163
  2. 2019A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithmscited by 142
  3. 2018Deep Variational Reinforcement Learning for POMDPscited by 125
  4. 2019End-to-End Robotic Reinforcement Learning without Reward Engineeringcited by 122
  5. 2016Interactive Perception: Leveraging Action in Perception and Perception in Actioncited by 78
  6. 2019ROBEL: Robotics Benchmarks for Learning with Low-Cost Robotscited by 72
  7. 2020Offline Learning from Demonstrations and Unlabeled Experiencecited by 72
  8. 2018Variational Inverse Control with Events: A General Framework for Data-Driven Reward Definitioncited by 69
  9. 2019Shaping Belief States with Generative Environment Models for RLcited by 66
  10. 2019Risk Averse Robust Adversarial Reinforcement Learningcited by 57
  11. 2018On Improving Deep Reinforcement Learning for POMDPscited by 50
  12. 2019Sim-to-(Multi)-Real: Transfer of Low-Level Robust Control Policies to Multiple Quadrotorscited by 45
  13. 2021Replacing Rewards with Examples: Example-Based Policy Search via Recursive Classificationcited by 38
  14. 2018Few-Shot Goal Inference for Visuomotor Learning and Planningcited by 30
  15. 2016Visual Closed-Loop Control for Pouring Liquidscited by 26
  16. 2016Collective Robot Reinforcement Learning with Distributed Asynchronous Guided Policy Searchcited by 25
  17. 2019Learning Belief Representations for Imitation Learning in POMDPscited by 24
  18. 2019Improved Robustness and Safety for Autonomous Vehicle Control with Adversarial Reinforcement Learningcited by 23
  19. 2020Semi-supervised reward learning for offline reinforcement learningcited by 23
  20. 2016Supervision via Competition: Robot Adversaries for Learning Taskscited by 16
  21. 2018Sample Efficient Deep Reinforcement Learning for Dialogue Systems with Large Action Spacescited by 15
  22. 2016Face valuing: Training user interfaces with facial expressions and reinforcement learningcited by 6

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Privileged Sensing Scaffolds Reinforcement Learningcited by 19