mlsanityData to 14 Oct 2026

On the Convergence of Success Conditioning for Policy Optimization

arXiv 2610.03642 ↗First posted 2 Oct 2026cs.LG math.OCcited by 0

Not on the map. Why?

Abstract (arXiv)

Success conditioning is a strategy for improving decision-making policies in stochastic environments; it updates a policy by increasing the probability of taking actions that yield successful outcomes. Success conditioning is common to many reinforcement learning applications, yet its limiting behavior and convergence rates are not well understood. In this work, we demonstrate that success conditioning converges to an optimal policy on a broad class of Markov decision processes (MDPs). We also derive convergence rates in some common settings. For discounted MDPs, we prove convergence within O(1/varepsilonp) iterations to an varepsilon-optimal policy, where the exponent p depends on problem data. For single-period MDPs, such a policy is obtained within O(log(1/varepsilon)) iterations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 21 references that are in the map. First posted on arXiv 2 Oct 2026.

Builds on

17 of 21 references.

References · 17

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2015Trust Region Policy Optimizationcited by 2,580
  5. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  6. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  7. 2018Hierarchical Neural Story Generationcited by 1,193
  8. 2017Hindsight Experience Replaycited by 1,136
  9. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
  10. 2019Goal-conditioned Imitation Learningcited by 162
  11. 2019Adaptive Trust Region Policy Optimization: Global Convergence and Faster Rates for Regularized MDPscited by 118
  12. 2022When does return-conditioned supervised learning work for offline reinforcement learning?cited by 76
5 more
  1. 2025Self-Improving Embodied Foundation Modelscited by 32
  2. 2022Imitating Past Successes can be Very Suboptimalcited by 23
  3. 2025On the Role of Temperature Sampling in Test-Time Scalingcited by 11
  4. 2021Reward-Weighted Regression Converges to a Global Optimumcited by 6
  5. 2026Success Conditioning as Policy Improvement: The Optimization Problem Solved by Imitating Successcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024On the Global Convergence of Policy Gradient in Average Reward Markov Decision Processes
  2. 2024Performative Reinforcement Learning with Linear Markov Decision Process
  3. 2025Non-stationary and Varying-discounting Markov Decision Processes for Reinforcement Learning
  4. 2024Reinforcement Learning for Infinite-Horizon Average-Reward Linear MDPs via Approximation by Discounted-Reward MDPs
  5. 2026Reinforcement Learning for Exponential Utility: Algorithms and Convergence in Discounted MDPs
  6. 2021Fast Rates for the Regret of Offline Reinforcement Learning
  7. 2026Towards Optimal Policy Improvement
  8. 2025Optimistically Optimistic Exploration for Provably Efficient Infinite-Horizon Reinforcement and Imitation Learning