mlsanityData to 14 Oct 2026

Improving Policy Optimization via varepsilon-Retrain

arXiv 2406.08315 ↗First posted 12 Jun 2024cs.AI cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

We present varepsilon-retrain, an exploration strategy encouraging a behavioral preference while optimizing policies with monotonic improvement guarantees. To this end, we introduce an iterative procedure for collecting retrain areas -- parts of the state space where an agent did not satisfy the behavioral preference. Our method switches between the typical uniform restart state distribution and the retrain areas using a decaying factor varepsilon, allowing agents to retrain on situations where they violated the preference. We also employ formal verification of neural networks to provably quantify the degree to which agents adhere to these behavioral preferences. Experiments over hundreds of seeds across locomotion, power network, and navigation tasks show that our method yields agents that exhibit significant performance and sample efficiency improvements.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 23 of its 39 references that are in the map. First posted on arXiv 12 Jun 2024.

Builds on

23 of 39 references.

References · 23

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Trust Region Policy Optimizationcited by 2,580
  3. 2016Concrete Problems in AI Safetycited by 1,642
  4. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 316
  5. 2020First return, then explorecited by 266
  6. 2019Algorithms for Verifying Deep Neural Networkscited by 221
  7. 2018Formal Security Analysis of Neural Networks using Symbolic Intervalscited by 185
  8. 2023Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmarkcited by 143
  9. 2017Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learningcited by 90
  10. 2023OmniSafe: An Infrastructure for Accelerating Safe Reinforcement Learning Researchcited by 80
  11. 2021Direct Behavior Specification via Constrained Reinforcement Learningcited by 31
  12. 2023On the Importance of Exploration for Generalization in Reinforcement Learningcited by 31
11 more
  1. 2019A Hitchhiker's Guide to Statistical Comparisons of Reinforcement Learning Algorithmscited by 27
  2. 2021Neural Network Repair with Reachability Analysiscited by 19
  3. 2023The #DNN-Verification Problem: Counting Unsafe Inputs for Deep Neural Networkscited by 19
  4. 2023Enumerating Safe Regions in Deep Neural Networks with Provable Probabilistic Guaranteescited by 19
  5. 2021Gym-ANM: Reinforcement Learning Environments for Active Network Management Tasks in Electricity Distribution Systemscited by 18
  6. 2018Curiosity-driven Exploration for Mapless Navigation with Deep Reinforcement Learningcited by 14
  7. 2021Centralizing State-Values in Dueling Networks for Multi-Robot Reinforcement Learning Mapless Navigationcited by 13
  8. 2023Safe Deep Reinforcement Learning by Verifying Task-Level Propertiescited by 9
  9. 2023Contrastive Initial State Buffer for Reinforcement Learningcited by 8
  10. 2023Online Safety Property Collection and Refinement for Safe Deep Reinforcement Learning in Mapless Navigationcited by 7
  11. 2023Improving Deep Policy Gradients with Value Function Searchcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 1.

5 papers.

  1. 2025RL2Grid: Benchmarking Reinforcement Learning in Power Grid Operationscited by 9
  2. 2024On Stateful Value Factorization in Multi-Agent Reinforcement Learningcited by 5
  3. 2025Designing Control Barrier Function via Probabilistic Enumeration for Safe Reinforcement Learning Navigationcited by 3
  4. 2025Probabilistically Tightened Linear Relaxation-based Perturbation Analysis for Neural Network Verificationcited by 2
  5. 2026Probabilistic Verification of Recurrent Neural Networks for Single and Multi-Agent Reinforcement Learningcited by 0