mlsanityData to 14 Oct 2026

Consensus Based Stochastic Control

arXiv 2501.17801 ↗First posted 29 Jan 2025math.OCcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

We propose a gradient-free deep reinforcement learning algorithm to solve high-dimensional, finite-horizon stochastic control problems. Although the recently developed deep reinforcement learning framework has achieved great success in solving these problems, direct estimation of policy gradients from Monte Carlo sampling often suffers from high variance. To address this, we introduce the Momentum Consensus-Based Optimization (M-CBO) and Adaptive Momentum Consensus-Based Optimization (Adam-CBO) frameworks. These methods optimize policies using Monte Carlo estimates of the value function, rather than its gradients. Adjustable Gaussian noise supports efficient exploration, helping the algorithm converge to optimal policies in complex, nonconvex environments. Numerical results confirm the accuracy and scalability of our approach across various problem dimensions and show the potential for extension to mean-field control problems. Theoretically, we prove that M-CBO can converge to the optimal policy under some assumptions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 40 of its 73 references that are in the map. First posted on arXiv 29 Jan 2025.

Builds on

40 of 73 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Model-based Reinforcement Learning for Parameterized Action Spaces3 Apr
  2. 2024Near-Optimal Reinforcement Learning with Self-Play under Adaptivity Constraints2 Feb

Other references · 38

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2015Trust Region Policy Optimizationcited by 2,580
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  6. 2017Solving high-dimensional partial differential equations using deep learningcited by 556
  7. 2017Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equationscited by 191
  8. 2017Emergence of Locomotion Behaviours in Rich Environmentscited by 165
  9. 2018Learning to Walk via Deep Reinforcement Learningcited by 158
  10. 2021Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithmscited by 101
  11. 2021Policy Evaluation and Temporal-Difference Learning in Continuous Time and Space: A Martingale Approachcited by 84
  12. 2020Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path spacecited by 82
26 more
  1. 2019Model-Free Mean-Field Reinforcement Learning: Mean-Field MDP and Mean-Field Q-Learningcited by 59
  2. 2020Mean-Field Controls with Q-learning for Cooperative MARL: Convergence and Complexity Analysiscited by 49
  3. 2018Nonparametric inference of interaction laws in systems of agents from trajectory datacited by 44
  4. 2024In value-based deep reinforcement learning, a pruned network is a good networkcited by 39
  5. 2022Stochastic Optimal Control for Collective Variable Free Sampling of Molecular Transition Pathscited by 38
  6. 2024Q-value Regularized Transformer for Offline Reinforcement Learningcited by 29
  7. 2023Recent Developments in Machine Learning Methods for Stochastic Control and Gamescited by 24
  8. 2016Mean Field Game of Controls and An Application To Trade Crowdingcited by 23
  9. 2019Neural networks-based backward scheme for fully nonlinear PDEscited by 23
  10. 2021Consensus-Based Optimization Methods Converge Globallycited by 16
  11. 2024The Max-Min Formulation of Multi-Objective Reinforcement Learning: From Theory to a Model-Free Algorithmcited by 15
  12. 2022Mean-field neural networks-based algorithms for McKean-Vlasov control problems *cited by 10
  13. 2024Constrained Reinforcement Learning Under Model Mismatchcited by 8
  14. 2024On Bellman equations for continuous-time policy evaluation I: discretization and approximationcited by 7
  15. 2013Efficiency of the Price Formation Process in Presence of High Frequency Participants: a Mean Field Game analysiscited by 6
  16. 2017Mean-field type modeling of nonlocal crowd aversion in pedestrian crowd dynamicscited by 6
  17. 2023rm E(3)-Equivariant Actor-Critic Methods for Cooperative Multi-Agent Reinforcement Learningcited by 6
  18. 2024Confidence Aware Inverse Constrained Reinforcement Learningcited by 6
  19. 2024RVI-SAC: Average Reward Off-Policy Deep Reinforcement Learningcited by 6
  20. 2021On the mean-field limit for the consensus-based optimizationcited by 5
  21. 2024A Primal-Dual Algorithm for Offline Constrained Reinforcement Learning with Linear MDPscited by 5
  22. 2020A consensus-based global optimization method with adaptive momentum estimationcited by 4
  23. 2023Accelerated Policy Gradient: On the Convergence Rates of the Nesterov Momentum for Reinforcement Learningcited by 4
  24. 2024An Efficient On-Policy Deep Learning Framework for Stochastic Optimal Controlcited by 3
  25. 2024Safe Reinforcement Learning using Finite-Horizon Gradient-based Estimationcited by 3
  26. 2023Probabilistic Constrained Reinforcement Learning with Formal Interpretabilitycited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Deterministic Policy Gradient for Reinforcement Learning with Continuous Time and State
  2. 2025Policy Gradient with Second Order Momentum
  3. 2025Controlling dynamics of stochastic systems with deep reinforcement learning
  4. 2024Deep Gaussian Covariance Network with Trajectory Sampling for Data-Efficient Policy Search
  5. 2025Adversarial Reinforcement Learning: A Duality-Based Approach To Solving Optimal Control Problems
  6. 2024Accelerated Multi-Time-Scale Stochastic Approximation: Optimal Complexity and Applications in Reinforcement Learning and Multi-Agent Games
  7. 2024The Definitive Guide to Policy Gradients in Deep Reinforcement Learning: Theory, Algorithms and Implementations
  8. 2024Learning Optimal Deterministic Policies with Stochastic Policy Gradients