mlsanityData to 7 Oct 2026

Deep Residual Reinforcement Learning

arXiv 1905.01072 ↗First posted 3 May 2019cs.LG cs.AI stat.MLcited by 21

Not on the map. Why?

Abstract (arXiv)

We revisit residual algorithms in both model-free and model-based reinforcement learning settings. We propose the bidirectional target network technique to stabilize residual algorithms, yielding a residual version of DDPG that significantly outperforms vanilla DDPG in the DeepMind Control Suite benchmark. Moreover, we find the residual algorithm an effective approach to the distribution mismatch problem in model-based planning. Compared with the existing TD(k) method, our residual-based method makes weaker assumptions about the model and yields a greater performance boost.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 5; 2022: 3; 2023: 3; 2024: 5; 2025: 2; 2026: 2.

21 papers.

Sort:

By area:

  1. 2022Learning GFlowNets from partial episodes for improved convergence and stabilitycited by 129
  2. 2023Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrationscited by 74
  3. 2024Policy Decorator: Model-Agnostic Online Refinement for Large Policy Modelcited by 60
  4. 2022Why Should I Trust You, Bellman? The Bellman Error is a Poor Replacement for Value Errorcited by 31
  5. 2024ODICE: Revealing the Mystery of Distribution Correction Estimation via Orthogonal-gradient Updatecited by 21
  6. 2020Average-reward model-free reinforcement learning: a systematic review and literature mappingcited by 19
  7. 2021A Deep Reinforcement Learning Approach to Marginalized Importance Sampling with the Successor Representationcited by 19
  8. 2025Touch begins where vision ends: Generalizable policies for contact-rich manipulationcited by 13
  9. 2021Correcting Momentum in Temporal Difference Learningcited by 12
  10. 2026RFS: Reinforcement Learning with Residual Flow Steering for Dexterous Manipulationcited by 10
  11. 2021Convergent and Efficient Deep Q Network Algorithmcited by 8
  12. 2021Self-Consistent Models and Valuescited by 7
9 more
  1. 2023Toward Efficient Gradient-Based Value Estimationcited by 4
  2. 2025An Optimal Discriminator Weighted Imitation Perspective for Reinforcement Learningcited by 4
  3. 2021Analysis and Optimisation of Bellman Residual Errors with Neural Function Approximationcited by 2
  4. 2022An Experimental Comparison Between Temporal Difference and Residual Gradient with Neural Network Approximationcited by 2
  5. 2023A new Gradient TD Algorithm with only One Step-size: Convergence Rate Analysis using L-λ Smoothnesscited by 2
  6. 2024DouRN: Improving DouZero by Residual Neural Networkscited by 0
  7. 2024Probing Implicit Bias in Semi-gradient Q-learning: Visualizing the Effective Loss Landscapes via the Fokker--Planck Equationcited by 0
  8. 2024Fast and Accurate Neural Rendering Using Semi-Gradientscited by 0
  9. 2026AnyBody: Free-Form Whole-Body Humanoid Control from Arbitrary Keypoint Guidancecited by 0