mlsanityData to 14 Oct 2026

Operator Splitting Value Iteration

arXiv 2211.13937 ↗First posted 25 Nov 2022cs.LG cs.AI cs.SYcited by 8

Not on the map. Why?

Abstract (arXiv)

We introduce new planning and reinforcement learning algorithms for discounted MDPs that utilize an approximate model of the environment to accelerate the convergence of the value function. Inspired by the splitting approach in numerical linear algebra, we introduce Operator Splitting Value Iteration (OS-VI) for both Policy Evaluation and Control problems. OS-VI achieves a much faster convergence rate when the model is accurate enough. We also introduce a sample-based version of the algorithm called OS-Dyna. Unlike the traditional Dyna architecture, OS-Dyna still converges to the correct value function in presence of model approximation error.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 16 of its 55 references that are in the map. First posted on arXiv 25 Nov 2022.

Builds on

16 of 55 references.

References · 16

  1. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  2. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  3. 2019Information-Theoretic Considerations in Batch Reinforcement Learningcited by 287
  4. 2020Model-Based Reinforcement Learning with Value-Targeted Regressioncited by 234
  5. 2019A Theoretical Analysis of Deep Q-Learningcited by 215
  6. 2017Value Prediction Networkcited by 117
  7. 2020Objective Mismatch in Model-based Reinforcement Learningcited by 96
  8. 2020The Value Equivalence Principle for Model-Based Reinforcement Learningcited by 94
  9. 2016The Predictron: End-To-End Learning and Planningcited by 93
  10. 2019Stochastic approximation with cone-contractive operators: Sharp ℓ_∞-bounds for Q-learningcited by 77
  11. 2021Control-Oriented Model-Based Reinforcement Learning with Implicit Differentiationcited by 35
  12. 2022Value Gradient weighted Model-Based Reinforcement Learningcited by 32
4 more
  1. 2020Policy-Aware Model Learning for Policy Gradient Methodscited by 23
  2. 2019Gradient-Aware Model-based Policy Searchcited by 19
  3. 2016Policy Error Bounds for Model-Based Reinforcement Learning with Factored Linear Modelscited by 5
  4. 2016A Matrix Splitting Perspective on Planning with Optionscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 2; 2025: 2; 2026: 1.

8 papers.

  1. 2024Deflated Dynamics Value Iterationcited by 5
  2. 2023From Optimization to Control: Quasi Policy Iterationcited by 4
  3. 2025Rank-One Modified Value Iterationcited by 3
  4. 2023Maximum Entropy Model Correction in Reinforcement Learningcited by 2
  5. 2024PID Accelerated Temporal Difference Algorithmscited by 2
  6. 2025Calibrated Value-Aware Model Learning with Probabilistic Environment Modelscited by 2
  7. 2023λ-models: Effective Decision-Aware Reinforcement Learning with Latent Modelscited by 1
  8. 2026Limiting-Kernel Q(λ): Bridging Short and Long Horizonscited by 0