mlsanityData to 7 Oct 2026

Transitive RL: Value Learning via Divide and Conquer

arXiv 2510.22512 ↗First posted 26 Oct 2025cs.LG cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

In this work, we present Transitive Reinforcement Learning (TRL), a new value learning algorithm based on a divide-and-conquer paradigm. TRL is designed for offline goal-conditioned reinforcement learning (GCRL) problems, where the aim is to find a policy that can reach any state from any other state in the smallest number of steps. TRL converts a triangle inequality structure present in GCRL into a practical divide-and-conquer value update rule. This has several advantages compared to alternative value learning paradigms. Compared to temporal difference (TD) methods, TRL suffers less from bias accumulation, as in principle it only requires O(log T) recursions (as opposed to O(T) in TD learning) to handle a length-T trajectory. Unlike Monte Carlo methods, TRL suffers less from high variance as it performs dynamic programming. Experimentally, we show that TRL achieves the best performance in highly challenging, long-horizon benchmark tasks compared to previous offline GCRL algorithms.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 45 of its 53 references that are in the map. First posted on arXiv 26 Oct 2025.

Builds on

45 of 53 references.

References · 45

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  3. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  4. 2015Continuous control with deep reinforcement learningcited by 3,279
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  6. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  7. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  8. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  9. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  10. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  11. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  12. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
33 more
  1. 2019Behavior Regularized Offline Reinforcement Learningcited by 654
  2. 2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policiescited by 307
  3. 2024Flow Matching Guide and Codecited by 291
  4. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  5. 2018Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimationcited by 226
  6. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 216
  7. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 188
  8. 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modelingcited by 177
  9. 2025Flow Q-Learningcited by 162
  10. 2021Offline RL Without Off-Policy Evaluationcited by 150
  11. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 133
  12. 2023Revisiting the Minimalist Approach to Offline Reinforcement Learningcited by 121
  13. 2021Rapid Exploration for Open-World Navigation with Latent Goal Modelscited by 118
  14. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 116
  15. 2023Offline RL with No OOD Actions: In-Sample Learning via Implicit Value Regularizationcited by 114
  16. 2020EMaQ: Expected-Max Q-Learning Operator for Simple Yet Effective Offline and Online RLcited by 113
  17. 2023Extreme Q-Learning: MaxEnt RL without Entropycited by 113
  18. 2020C-Learning: Learning to Achieve Goals via Recursive Classificationcited by 99
  19. 2022Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RLcited by 89
  20. 2024Foundation Policies with Hilbert Representationscited by 89
  21. 2024Is Value Learning Really the Main Bottleneck in Offline RL?cited by 82
  22. 2021Learning Successor States and Goal-Dependent Values: A Mathematical Viewpointcited by 67
  23. 2023Reinforcement Learning from Passive Data via Latent Intentionscited by 64
  24. 2025Horizon Reduction Makes RL Scalablecited by 55
  25. 2024Learning Temporal Distances: Contrastive Successor Features Can Provide a Metric Structure for Decision-Makingcited by 53
  26. 2020Model-Based Visual Planning with Self-Supervised Functional Distancescited by 41
  27. 2023Contrastive Difference Predictive Codingcited by 35
  28. 2020Sub-Goal Trees -- a Framework for Goal-Based Reinforcement Learningcited by 27
  29. 2020Divide-and-Conquer Monte Carlo Tree Search For Goal-Directed Planningcited by 19
  30. 2021Solving Compositional Reinforcement Learning Problems via Task Reductioncited by 19
  31. 2023Distance Weighted Supervised Learning for Offline Interaction Datacited by 18
  32. 2025Behavior-Regularized Diffusion Policy Optimization for Offline Reinforcement Learningcited by 18
  33. 2018Floyd-Warshall Reinforcement Learning: Learning from Past Experiences to Reach New Goalscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 12.

12 papers.

  1. 2026Chunk-Guided Q-Learningcited by 3
  2. 2026Refining Compositional Diffusion for Reliable Long-Horizon Planningcited by 2
  3. 2026Abstraction for Offline Goal-Conditioned Reinforcement Learningcited by 2
  4. 2026Can We Really Learn One Representation to Optimize All Rewards?cited by 1
  5. 2026Mollified Value Learningcited by 1
  6. 2026Offline Reinforcement Learning with Universal Horizon Modelscited by 1
  7. 2026Adaptive Coarse-to-Fine Subgoal Refinement for Long-Horizon Offline Goal-Conditioned Reinforcement Learningcited by 1
  8. 2026Recursive Value Learning for Long-Horizon Offline Goal-Conditioned RLcited by 1
  9. 2026Zero-Shot Signal Temporal Logic Planning with Disjunctive Branch Selection in Dynamic Semantic Mapscited by 0
  10. 2026ChronoForest: Closed-Loop Multi-Tree Diffusion Planning for Efficient Bridge Search and Route Compositioncited by 0
  11. 2026SegBench-GC: Testing Segmentation Invariance in Multi-Step Offline Goal-Conditioned Reinforcement Learningcited by 0
  12. 2026GTRL: Grounding Divide-and-Conquer Value Learning with Temporal Differencescited by 0