mlsanityData to 14 Oct 2026

Constant in an Ever-Changing World

arXiv 2510.03330 ↗First posted 1 Oct 2025cs.LGcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

The training process of reinforcement learning often suffers from severe oscillations, leading to instability and degraded performance. In this paper, we propose a Constant in an Ever-Changing World (CIC) framework that enhances algorithmic stability to improve performance. CIC maintains both a representative policy and a current policy. Instead of updating the representative policy blindly, CIC selectively updates it only when the current policy demonstrates superiority. Furthermore, CIC employs an adaptive adjustment mechanism, enabling the representative and current policies to jointly facilitate critic training. We evaluate CIC on five MuJoCo environments, and the results show that CIC improves the performance of conventional algorithms without incurring additional computational cost.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 5 of its 8 references that are in the map. First posted on arXiv 1 Oct 2025.

Builds on

5 of 8 references.

References · 5

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  5. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 288

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Stabilizing Reinforcement Learning with LLMs: Formulation and Practices
  2. 2026Beyond Single-Model Optimization: Preserving Plasticity in Continual Reinforcement Learning
  3. 2025EXPO: Stable Reinforcement Learning with Expressive Policies
  4. 2025Continual Reinforcement Learning by Planning with Online World Models
  5. 2026Position: Deployed Reinforcement Learning should be Continual
  6. 2026Unlocking the Critic: Reward-Free Policy Optimization for LLM Post-Training
  7. 2026PCSD: Persistent Consistency for Self-Distillation in Agentic Reinforcement Learning
  8. 2024Pausing Policy Learning in Non-stationary Reinforcement Learning