mlsanityData to 14 Oct 2026

Matrix-Space Reinforcement Learning for Reusing Local Transition Geometry

arXiv 2605.14304 ↗First posted 14 May 2026cs.LG cs.AIcited by 2

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Compositional generalization in sequential decision-making requires identifying which parts of prior rollouts remain useful for new tasks. Existing methods reuse skills or predictive models, but often overlook rich local transition geometry and dynamics. We propose Matrix-Space Reinforcement Learning (MSRL), a geometric abstraction that represents trajectory segments through positive semidefinite matrix descriptors aggregating first- and second-order statistics of lifted one-step transitions. These descriptors expose shared hidden structure, support algebraic composition in an abstract matrix space, and reveal opportunities for transfer. We prove that the descriptor is well defined up to coordinate gauge, complete for the induced low-order additive signal class, additive under valid segment composition, and minimally sufficient among admissible additive descriptors. We further show that conditioning value functions on the trajectory-segment matrix yields a first-order smooth approximation of action values, enabling source-learned matrix-to-value mappings to bootstrap learning in new tasks. MSRL is plug-in compatible with standard model-free and model-based methods, while obstruction filtering rejects implausible compositions. Empirically, MSRL achieves the best average finite-budget target AUC of 0.73, outperforming MSRL from scratch (0.65), TD-MPC-PT+FT (0.63), and TD-MPC (0.57).

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 30 of its 54 references that are in the map. First posted on arXiv 14 May 2026.

Builds on

30 of 54 references.

References · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  5. 2018How Powerful are Graph Neural Networks?cited by 4,393
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  7. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  8. 2016Progressive Neural Networkscited by 1,361
  9. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  10. 2017Hindsight Experience Replaycited by 1,136
  11. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  12. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
18 more
  1. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  2. 2016Kernel Mean Embedding of Distributions: A Review and Beyondcited by 364
  3. 2016Successor Features for Transfer in Reinforcement Learningcited by 340
  4. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 327
  5. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 207
  6. 2019Scalable methods for computing state similarity in deterministic Markov Decision Processescited by 133
  7. 2019Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvementcited by 88
  8. 2017Eigenoption Discovery through the Deep Successor Representationcited by 86
  9. 2021The Option Keyboard: Combining Skills in Reinforcement Learningcited by 67
  10. 2017Near Optimal Behavior via Approximate State Abstractioncited by 66
  11. 2024Modeling Other Players with Bayesian Beliefs for Games with Incomplete Informationcited by 17
  12. 2025Network Diffuser for Placing-Scheduling Service Function Chains with Inverse Demonstrationcited by 11
  13. 2026Geometry of Drifting MDPs with Path-Integral Stability Certificatescited by 8
  14. 2026Manifold-Constrained Energy-Based Transition Models for Offline Reinforcement Learningcited by 8
  15. 2026Structuring Value Representations via Geometric Coherence in Markov Decision Processescited by 6
  16. 2026Cochain Perspectives on Temporal-Difference Signals for Learning Beyond Markov Dynamicscited by 5
  17. 2026NonZero: Interaction-Guided Exploration for Multi-Agent Monte Carlo Tree Searchcited by 5
  18. 2026Operator-Guided Invariance Learning for Continuous Reinforcement Learningcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Minimal Markovization via Stable Quotients in Holonomy-Cover Decision Processescited by 2
  2. 2026Geometry-Aware Operator Families for Structured Representation Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Geometry of Neural Reinforcement Learning in Continuous State and Action Spaces
  2. 2026Factorized Spectral Representations for Reinforcement Learning
  3. 2024Geometric Fabrics: a Safe Guiding Medium for Policy Learning
  4. 2026Hitting Time Isomorphism for Multi-Stage Planning with Foundation Policies
  5. 2025A Primer on SO(3) Action Representations in Deep Reinforcement Learning
  6. 2026Improving and Accelerating Offline RL in Large Discrete Action Spaces with Structured Policy Initialization
  7. 2022Multi-Task Imitation Learning for Linear Dynamical Systems
  8. 2026The Terminal Representation in Reinforcement Learning