mlsanityData to 7 Oct 2026

L2C2: Locally Lipschitz Continuous Constraint towards Stable and Smooth Reinforcement Learning

arXiv 2202.07152 ↗First posted 15 Feb 2022cs.RO cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

This paper proposes a new regularization technique for reinforcement learning (RL) towards making policy and value functions smooth and stable. RL is known for the instability of the learning process and the sensitivity of the acquired policy to noise. Several methods have been proposed to resolve these problems, and in summary, the smoothness of policy and value functions learned mainly in RL contributes to these problems. However, if these functions are extremely smooth, their expressiveness would be lost, resulting in not obtaining the global optimal solution. This paper therefore considers RL under local Lipschitz continuity constraint, so-called L2C2. By designing the spatio-temporal locally compact space for L2C2 from the state transition at each time step, the moderate smoothness can be achieved without loss of expressiveness. Numerical noisy simulations verified that the proposed L2C2 outperforms the task performance while smoothing out the robot action generated from the learned policy.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 24 of its 40 references that are in the map. First posted on arXiv 15 Feb 2022.

Builds on

24 of 40 references.

References · 24

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2016Layer Normalizationcited by 5,579
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  4. 2016OpenAI Gymcited by 1,684
  5. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  6. 2015Prioritized Experience Replaycited by 1,070
  7. 2017Constrained Policy Optimizationcited by 928
  8. 2016Deep Exploration via Bootstrapped DQNcited by 518
  9. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 413
  10. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 395
  11. 2018Quantifying Generalization in Reinforcement Learningcited by 345
  12. 2018Regularisation of Neural Networks by Enforcing Lipschitz Continuitycited by 311
12 more
  1. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 235
  2. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 156
  3. 2019A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithmscited by 136
  4. 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 133
  5. 2016Connecting Generative Adversarial Networks and Actor-Critic Methodscited by 49
  6. 2020Regularizing Action Policies for Smooth Control with Reinforcement Learningcited by 47
  7. 2021Squareplus: A Softplus-Like Algebraic Rectifiercited by 18
  8. 2020t-Soft Update of Target Network for Deep Reinforcement Learningcited by 16
  9. 2020Proximal Policy Optimization with Relative Pearson Divergencecited by 9
  10. 2020How to Train your Quadrotor: A Framework for Consistently Smooth and Responsive Flight Control via Reinforcement Learningcited by 5
  11. 2018Temporal Regularization in Markov Decision Processcited by 3
  12. 2021Coarse-Grained Smoothness for RL in Metric Spacescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 1; 2024: 6; 2025: 5; 2026: 8.

22 papers.

Sort:

By area:

  1. 2025Learning Humanoid Standing-up Control across Diverse Posturescited by 34
  2. 2024Learning Smooth Humanoid Locomotion through Lipschitz-Constrained Policiescited by 26
  3. 2025PhysHSI: Towards a Real-World Generalizable and Natural Humanoid-Scene Interaction Systemcited by 17
  4. 2022Consolidated Adaptive T-soft Update for Deep Reinforcement Learningcited by 5
  5. 2022Reward Bonuses with Gain Scheduling Inspired by Iterative Deepening Searchcited by 5
  6. 2024Benchmarking Smoothness and Reducing High-Frequency Oscillations in Continuous Control Policiescited by 5
  7. 2024Revisiting Experience Replayable Conditionscited by 3
  8. 2023Soft Actor-Critic Algorithm with Truly-satisfied Inequality Constraintcited by 2
  9. 2026Enhancing Control Policy Smoothness by Aligning Actions with Predictions from Preceding Statescited by 2
  10. 2024HG2P: Hippocampus-inspired High-reward Graph and Model-Free Q-Gradient Penalty for Path Planning and Motion Controlcited by 1
  11. 2024Weber-Fechner Law in Temporal Difference learning derived from Control as Inferencecited by 1
  12. 2025Implicit Constraint-Aware Off-Policy Correction for Offline Reinforcement Learningcited by 1
10 more
  1. 2026Stabilizing the Q-Gradient Field for Policy Smoothness in Actor-Critic Methodscited by 1
  2. 2024Lipschitz-Regularized Critics Lead to Policy Robustness Against Transition Dynamics Uncertaintycited by 0
  3. 2025Deep Reinforcement Learning for Day-to-day Dynamic Tolling in Tradable Credit Schemescited by 0
  4. 2025Temporally smoothed incremental model-based heuristic dynamic programming for command-filtered cascaded online learning flight controlcited by 0
  5. 2026CMR: Contractive Mapping Embeddings for Robust Humanoid Locomotion on Unstructured Terrainscited by 0
  6. 2026Beyond Coefficients: Forecast-Necessity Testing for Interpretable Causal Discovery in Nonlinear Time-Series Modelscited by 0
  7. 2026Implicit Action Chunking for Smooth Continuous Controlcited by 0
  8. 2026Difference-Aware Retrieval Policies for Imitation Learningcited by 0
  9. 2026Improving Generalization and Robustness in Offline Reinforcement Learning via Boundary-Aware Data Augmentationcited by 0
  10. 2026DAMPER: Return-Prioritized Gradient Control for Smooth Policiescited by 0