Free Random Projection for In-Context Reinforcement Learning

arXiv 2504.06983 ↗First posted 9 Apr 2025cs.LG math.PR stat.MLcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Hierarchical inductive biases are hypothesized to promote generalizable policies in reinforcement learning, as demonstrated by explicit hyperbolic latent representations and architectures. Therefore, a more flexible approach is to have these biases emerge naturally from the algorithm. We introduce Free Random Projection, an input mapping grounded in free probability theory that constructs random orthogonal matrices where hierarchical structure arises inherently. The free random projection integrates seamlessly into existing in-context reinforcement learning frameworks by encoding hierarchical organization within the input space without requiring explicit architectural modifications. Empirical results on multi-environment benchmarks show that free random projection consistently outperforms the standard random projection, leading to improvements in generalization. Furthermore, analyses within linearly solvable Markov decision processes and investigations of the spectrum of kernel random matrices reveal the theoretical underpinnings of free random projection's enhanced performance, highlighting its capacity for effective adaptation in hierarchically structured state spaces.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 40 of its 81 references that are in the map. First posted on arXiv 9 Apr 2025.

Builds on

40 of 81 references.

Closest ancestors · 1

Same direction, newest first

2024SPQR: Controlling Q-ensemble Independence with Spiked Random Model for Reinforcement Learning6 Jan

Other references · 39

2017Proximal Policy Optimization Algorithmscited by 15,215
2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
2018Exploration by Random Network Distillationcited by 993
2022Simplified State Space Layers for Sequence Modelingcited by 788
2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 763
2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 684
2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 672
2017The Implicit Bias of Gradient Descent on Separable Datacited by 655
2019Implicit Regularization in Deep Matrix Factorizationcited by 405
2019Hyperbolic Graph Convolutional Neural Networkscited by 402
2020The Pitfalls of Simplicity Bias in Neural Networkscited by 351

27 more

2019Gradient Descent Maximizes the Margin of Homogeneous Neural Networkscited by 320
2018Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometrycited by 265
2018Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learningcited by 215
2022In-context Reinforcement Learning with Algorithm Distillationcited by 190
2018Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networkscited by 176
2020Benign overfitting in ridge regressioncited by 153
2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 139
2017Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practicecited by 132
2023Structured State Space Models for In-Context Reinforcement Learningcited by 111
2018A Kernel Theory of Modern Data Augmentationcited by 106
2022General-Purpose In-Context Learning by Meta-Learning Transformerscited by 105
2023Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretrainingcited by 83
2020Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networkscited by 77
2018The Emergence of Spectral Universality in Deep Networkscited by 74
2013Divide and Conquer Kernel Ridge Regression: A Distributed Algorithm with Minimax Optimal Ratescited by 65
2023POPGym: Benchmarking Partially Observable Reinforcement Learningcited by 62
2018Products of Many Large Random Matrices and Gradients in Deep Neural Networkscited by 49
2022Hyperbolic Deep Reinforcement Learningcited by 25
2023In-Context Reinforcement Learning for Variable Action Spacescited by 24
2021Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networkscited by 21
2023Self-Supervised Reinforcement Learning that Transfers using Random Featurescited by 11
2023Asymptotically free sketched ridge ensembles: Risks, cross-validation, and tuningcited by 11
2020The Spectrum of Fisher Information of Deep Networks Achieving Dynamical Isometrycited by 9
2024Analysing Multi-Task Regression via Random Matrix Theory with Application to Time Series Forecastingcited by 8
2021Entangled Kernels -- Beyond Separabilitycited by 7
2021Free Probability for predicting the performance of feed-forward fully connected neural networkscited by 3
2018Cauchy noise loss for stochastic optimization of random matrix models via free deterministic equivalentscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way