mlsanityData to 14 Oct 2026

Variational Inverse Control with Events: A General Framework for Data-Driven Reward Definition

arXiv 1805.11686 ↗First posted 29 May 2018cs.LG stat.MLcited by 69

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

The design of a reward function often poses a major practical challenge to real-world applications of reinforcement learning. Approaches such as inverse reinforcement learning attempt to overcome this challenge, but require expert demonstrations, which can be difficult or expensive to obtain in practice. We propose variational inverse control with events (VICE), which generalizes inverse reinforcement learning methods to cases where full demonstrations are not needed, such as when only samples of desired goal states are available. Our method is grounded in an alternative perspective on control and reinforcement learning, where an agent's goal is to maximize the probability that one or more events will happen at some point in the future, rather than maximizing cumulative rewards. We demonstrate the effectiveness of our methods on continuous control tasks, with a focus on high-dimensional observations like images where rewards are hard or even impossible to specify.

Key facts. Cited by 69 papers in the map (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. First posted on arXiv 29 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 18; 2022: 13; 2023: 15; 2024: 16; 2025: 1; 2026: 5.

69 papers.

Sort:

By area:

  1. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 302
  2. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  3. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 176
  4. 2024RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedbackcited by 156
  5. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learningcited by 136
  6. 2023RoboCLIP: One Demonstration is Enough to Learn Robot Policiescited by 134
  7. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 119
  8. 2021Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationcited by 118
  9. 2021Learning by Watching: Physical Imitation of Manipulation Skills from Human Videoscited by 87
  10. 2022Can Foundation Models Perform Zero-Shot Task Specification For Robot Manipulation?cited by 70
  11. 2021Conservative Data Sharing for Multi-Task Offline Reinforcement Learningcited by 64
  12. 2022How to Leverage Unlabeled Data in Offline Reinforcement Learningcited by 62
38 more
  1. 2026Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisonscited by 60
  2. 2023Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learningcited by 54
  3. 2021Replacing Rewards with Examples: Example-Based Policy Search via Recursive Classificationcited by 38
  4. 2023Learning Zero-Shot Cooperation with Humans, Assuming Humans Are Biasedcited by 37
  5. 2021Skill Preferences: Learning to Extract and Execute Robotic Skills from Human Feedbackcited by 35
  6. 2021Lifelong Robotic Reinforcement Learning by Retaining Experiencescited by 34
  7. 2023Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learningcited by 34
  8. 2024Rank2Reward: Learning Shaped Reward Functions from Passive Videocited by 32
  9. 2021MURAL: Meta-Learning Uncertainty-Aware Rewards for Outcome-Driven Reinforcement Learningcited by 24
  10. 2023ZSC-Eval: An Evaluation Toolkit and Benchmark for Multi-agent Zero-shot Coordinationcited by 24
  11. 2022Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidancecited by 23
  12. 2024Affordance-Guided Reinforcement Learning via Visual Promptingcited by 23
  13. 2022A State-Distribution Matching Approach to Non-Episodic Reinforcement Learningcited by 22
  14. 2024Continuously Improving Mobile Manipulation with Autonomous Real-World RLcited by 22
  15. 2024Privileged Sensing Scaffolds Reinforcement Learningcited by 19
  16. 2023MAHALO: Unifying Offline Reinforcement Learning and Imitation Learning from Observationscited by 18
  17. 2021Outcome-Driven Reinforcement Learning via Variational Inferencecited by 17
  18. 2021Computational Benefits of Intermediate Rewards for Goal-Reaching Policy Learningcited by 17
  19. 2022Understanding the Complexity Gains of Single-Task RL with a Curriculumcited by 17
  20. 2025Rapidly Adapting Policies to the Real World via Simulation-Guided Fine-Tuningcited by 17
  21. 2022Inducing Structure in Reward Learning by Learning Featurescited by 16
  22. 2023Outcome-directed Reinforcement Learning by Uncertainty & Temporal Distance-Aware Curriculum Goal Generationcited by 15
  23. 2023REBOOT: Reuse Data for Bootstrapping Efficient Real-World Dexterous Manipulationcited by 15
  24. 2024Solving Minimum-Cost Reach Avoid using Reinforcement Learningcited by 14
  25. 2021Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Taskscited by 13
  26. 2022TarGF: Learning Target Gradient Field to Rearrange Objects without Explicit Goal Specificationcited by 13
  27. 2022Learning on the Job: Self-Rewarding Offline-to-Online Finetuning for Industrial Insertion of Novel Connectors from Visioncited by 13
  28. 2023Autonomous Robotic Reinforcement Learning with Asynchronous Human Feedbackcited by 13
  29. 2021Learning Perceptual Concepts by Bootstrapping from Human Queriescited by 12
  30. 2022Automating Reinforcement Learning with Example-based Resetscited by 12
  31. 2023Aligning Robot and Human Representationscited by 11
  32. 2022Learning from Guided Play: Improving Exploration for Adversarial Imitation Learning with Simple Auxiliary Taskscited by 10
  33. 2024DrS: Learning Reusable Dense Rewards for Multi-Stage Taskscited by 10
  34. 2024Adapt2Reward: Adapting Video-Language Models to Generalizable Robotic Rewards via Failure Promptscited by 10
  35. 2021DisCo RL: Distribution-Conditioned Reinforcement Learning for General-Purpose Policiescited by 9
  36. 2023CLIP-Motion: Learning Reward Functions for Robotic Actions Using Consecutive Observationscited by 9
  37. 2024On-Robot Reinforcement Learning with Goal-Contrastive Rewardscited by 8
  38. 2019Integration of Imitation Learning using GAIL and Reinforcement Learning using Task-achievement Rewards via Probabilistic Graphical Modelcited by 7

50 most cited of 69 All 69 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Reward Compatibility: A Framework for Inverse RL
  2. 2020Inverse Reinforcement Learning from a Gradient-based Learner
  3. 2025Generalizing Behavior via Inverse Reinforcement Learning with Closed-Form Reward Centroids
  4. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variables
  5. 2024Toward Information Theoretic Active Inverse Reinforcement Learning
  6. 2026Reward Transfer from Inverse Reinforcement Learning: A Coupled Minimax Approach
  7. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress
  8. 2023BC-IRL: Learning Generalizable Reward Functions from Demonstrations