mlsanityData to 7 Oct 2026

Safe CoR: A Dual-Expert Approach to Integrating Imitation Learning and Safe Reinforcement Learning Using Constraint Rewards

arXiv 2407.02245 ↗First posted 2 Jul 2024cs.RO cs.AIcited by 1

On the map: Offline RL and imitation learning from data → Safe offline RL

Abstract (arXiv)

In the realm of autonomous agents, ensuring safety and reliability in complex and dynamic environments remains a paramount challenge. Safe reinforcement learning addresses these concerns by introducing safety constraints, but still faces challenges in navigating intricate environments such as complex driving situations. To overcome these challenges, we present the safe constraint reward (Safe CoR) framework, a novel method that utilizes two types of expert demonstrationsunicodex2013reward expert demonstrations focusing on performance optimization and safe expert demonstrations prioritizing safety. By exploiting a constraint reward (CoR), our framework guides the agent to balance performance goals of reward sum with safety constraints. We test the proposed framework in diverse environments, including the safety gym, metadrive, and the realunicodex2013world Jackal platform. Our proposed framework enhances the performance of algorithms by 39% and reduces constraint violations by 88% on the real-world Jackal platform, demonstrating the framework's efficacy. Through this innovative approach, we expect significant advancements in real-world performance, leading to transformative effects in the realm of safe and reliable autonomous agents.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Safe offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 17 of its 25 references that are in the map. First posted on arXiv 2 Jul 2024.

Builds on

17 of 25 references.

References · 17

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2017Constrained Policy Optimizationcited by 928
  6. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  7. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  8. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  9. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 281
  10. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 124
  11. 2023Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priorscited by 36
  12. 2021Rule-based Optimal Control for Autonomous Drivingcited by 21
5 more
  1. 2023Trust Region-Based Safe Distributional Reinforcement Learning for Multiple Constraintscited by 20
  2. 2019Learning a Decision Module by Imitating Driver's Control Behaviorscited by 17
  3. 2023Efficient Off-Policy Safe Reinforcement Learning Using Trust Region Conditional Value at Riskcited by 11
  4. 2023TRC: Trust Region Conditional Value at Risk for Safe Reinforcement Learningcited by 4
  5. 2021Semi-Supervised Imitation Learning with Mixed Qualities of Demonstrations for Autonomous Drivingcited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Proactive Constrained Policy Optimization with Preemptive Penaltycited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025DIAL: Distribution-Informed Adaptive Learning of Multi-Task Constraints for Safety-Critical Systems
  2. 2023Learning Safety Constraints from Demonstrations with Unknown Rewards
  3. 2024Latent Safety-Constrained Policy Approach for Safe Offline Reinforcement Learning
  4. 2023Learning Shared Safety Constraints from Multi-task Demonstrations
  5. 2026OSIL: Learning Offline Safe Imitation Policies with Safety Inferred from Non-preferred Trajectories
  6. 2026Safe Score Matching: Diffusion Policies with Hamilton-Jacobi Reachability for Online Safe Reinforcement Learning
  7. 2025SafeWork-R1: Coevolving Safety and Intelligence under the AI-45circ Law
  8. 2025SafeMIL: Learning Offline Safe Imitation Policy from Non-Preferred Trajectories