mlsanityData to 14 Oct 2026

Temporal Difference Learning with Continuous Time and State in the Stochastic Setting

arXiv 2202.07960 ↗First posted 16 Feb 2022cs.LG cs.AI math.APcited by 5

Not on the map. Why?

Abstract (arXiv)

We consider the problem of continuous-time policy evaluation. This consists in learning through observations the value function associated with an uncontrolled continuous-time stochastic dynamic and a reward function. We propose two original variants of the well-known TD(0) method using vanishing time steps. One is model-free and the other is model-based. For both methods, we prove theoretical convergence rates that we subsequently verify through numerical simulations. Alternatively, those methods can be interpreted as novel reinforcement learning approaches for approximating solutions of linear PDEs (partial differential equations) or linear BSDEs (backward stochastic differential equations).

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 13 of its 49 references that are in the map. First posted on arXiv 16 Feb 2022.

Builds on

13 of 49 references.

References · 13

  1. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,150
  2. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 678
  3. 2017Solving high-dimensional partial differential equations using deep learningcited by 556
  4. 2017Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equationscited by 191
  5. 2013Non-strongly-convex smooth stochastic approximation with convergence rate O(1/n)cited by 111
  6. 2017Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equationscited by 65
  7. 2020An overview on deep learning-based approximation methods for partial differential equationscited by 65
  8. 2019Making Deep Q-learning methods robust to time discretizationcited by 58
  9. 2016Deep Learning Approximation for Stochastic Control Problemscited by 55
  10. 2021Continuous-Time Model-Based Reinforcement Learningcited by 41
  11. 2021Neural networks-based algorithms for stochastic control and PDEs in financecited by 26
  12. 2021Value Iteration in Continuous Actions, States and Timecited by 18
1 more
  1. 2018Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applicationscited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 2; 2026: 1.

5 papers.

  1. 2023Recent Developments in Machine Learning Methods for Stochastic Control and Gamescited by 24
  2. 2024On Bellman equations for continuous-time policy evaluation I: discretization and approximationcited by 7
  3. 2025Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Controlcited by 3
  4. 2025A Temporal Difference Method for Stochastic Continuous Dynamicscited by 3
  5. 2026Fast and Robust Convergence Rate for TD(0) with Linear Function Approximation, Universal Learning Steps and I.I.D. Samplescited by 0