Kalman Temporal Differences
Not on the map. Why?
Abstract (arXiv)
Because reinforcement learning suffers from a lack of scalability, online value (and Q-) function approximation has received increasing interest this last decade. This contribution introduces a novel approximation scheme, namely the Kalman Temporal Differences (KTD) framework, that exhibits the following features: sample-efficiency, non-linear approximation, non-stationarity handling and uncertainty management. A first KTD-based algorithm is provided for deterministic Markov Decision Processes (MDP) which produces biased estimates in the case of stochastic transitions. Than the eXtended KTD framework (XKTD), solving stochastic MDP, is described. Convergence is analyzed for special cases for both deterministic and stochastic transitions. Related algorithms are experimented on classical benchmarks. They compare favorably to the state of the art while exhibiting the announced features.
Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). First posted on arXiv 16 Jan 2014.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 3; 2022: 1; 2023: 1; 2024: 3; 2025: 0; 2026: 2.
10 papers.
- 2024Uncertainty-based Offline Variational Bayesian Reinforcement Learning for Robustness under Diverse Data Corruptionscited by 8
- 2021Don't Do What Doesn't Matter: Intrinsic Motivation with Action Usefulnesscited by 7
- 2023Toward Efficient Gradient-Based Value Estimationcited by 4
- 2024Fast Value Tracking for Deep Reinforcement Learningcited by 2
- 2024Lifelong Reinforcement Learning via Neuromodulationcited by 2
- 2021Multi-Agent Reinforcement Learning via Adaptive Kalman Temporal Difference and Successor Representationcited by 1
- 2022AKF-SR: Adaptive Kalman Filtering-based Successor Representationcited by 1
- 2021Findings from Experiments of On-line Joint Reinforcement Learning of Semantic Parser and Dialogue Manager with real Userscited by 0
- 2026Bayesian learning for the stochastic shortest path problemcited by 0
- 2026Generalized Kalman filter based temporal difference reinforcement learningcited by 0
- 2026Generalized Kalman filter based temporal difference reinforcement learning22 Jul 2026
- 2026Bayesian learning for the stochastic shortest path problem3 Jun 2026
- 2024Uncertainty-based Offline Variational Bayesian Reinforcement Learning for Robustness under Diverse Data Corruptions1 Nov 2024
- 2024Lifelong Reinforcement Learning via Neuromodulation15 Aug 2024
- 2024Fast Value Tracking for Deep Reinforcement Learning19 Mar 2024
- 2023Toward Efficient Gradient-Based Value Estimation31 Jan 2023
- 2022AKF-SR: Adaptive Kalman Filtering-based Successor Representation31 Mar 2022
- 2021Multi-Agent Reinforcement Learning via Adaptive Kalman Temporal Difference and Successor Representation30 Dec 2021
- 2021Findings from Experiments of On-line Joint Reinforcement Learning of Semantic Parser and Dialogue Manager with real Users25 Oct 2021
- 2021Don't Do What Doesn't Matter: Intrinsic Motivation with Action Usefulness20 May 2021