mlsanityData to 7 Oct 2026

Gradient Boosting Reinforcement Learning

arXiv 2407.08250 ↗First posted 11 Jul 2024cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

We present Gradient Boosting Reinforcement Learning (GBRL), a framework that adapts the strengths of gradient boosting trees (GBT) to reinforcement learning (RL) tasks. While neural networks (NNs) have become the de facto choice for RL, they face significant challenges with structured and categorical features and tend to generalize poorly to out-of-distribution samples. These are challenges for which GBTs have traditionally excelled in supervised learning. However, GBT's application in RL has been limited. The design of traditional GBT libraries is optimized for static datasets with fixed labels, making them incompatible with RL's dynamic nature, where both state distributions and reward signals evolve during training. GBRL overcomes this limitation by continuously interleaving tree construction with environment interaction. Through extensive experiments, we demonstrate that GBRL outperforms NNs in domains with structured observations and categorical features while maintaining competitive performance on standard continuous control benchmarks. Like its supervised learning counterpart, GBRL demonstrates superior robustness to out-of-distribution samples and better handles irregular state-action relationships.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 46 of its 77 references that are in the map. First posted on arXiv 11 Jul 2024.

Builds on

46 of 77 references.

References · 46

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  3. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2015Continuous control with deep reinforcement learningcited by 3,279
  6. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  7. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  8. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  9. 2016An overview of gradient descent optimization algorithmscited by 1,021
  10. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
  11. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  12. 2017Deep Reinforcement Learning that Matterscited by 727
34 more
  1. 2021Revisiting Deep Learning Models for Tabular Datacited by 725
  2. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 643
  3. 2019TabNet: Attentive Interpretable Tabular Learningcited by 640
  4. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
  5. 2021Tabular Data: Deep Learning is Not All You Needcited by 530
  6. 2021Deep Neural Networks and Tabular Data: A Surveycited by 489
  7. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 320
  8. 2021SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Trainingcited by 310
  9. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 257
  10. 2020How Neural Networks Extrapolate: From Feedforward to Graph Neural Networkscited by 218
  11. 2023When Do Neural Nets Outperform Boosted Trees on Tabular Data?cited by 218
  12. 2022Why do tree-based models still outperform deep learning on tabular data?cited by 198
  13. 2022On Embeddings for Numerical Features in Tabular Deep Learningcited by 185
  14. 2018Verifiable Reinforcement Learning via Policy Extractioncited by 176
  15. 2020What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Studycited by 157
  16. 2021Well-tuned Simple Nets Excel on Tabular Datasetscited by 151
  17. 2019NGBoost: Natural Gradient Boosting for Probabilistic Predictioncited by 94
  18. 2016A Deep Hierarchical Approach to Lifelong Learning in Minecraftcited by 90
  19. 2019Towards neural networks that provably know when they don't knowcited by 69
  20. 2020Uncertainty in Gradient Boosting via Ensemblescited by 56
  21. 2019Certified Adversarial Robustness for Deep Reinforcement Learningcited by 47
  22. 2021Boost then Convolve: Gradient Boosting Meets Graph Neural Networkscited by 42
  23. 2021Training Larger Networks for Deep Reinforcement Learningcited by 37
  24. 2020Know Your Limits: Uncertainty Estimation with ReLU Classifiers Fails at Reliable OOD Detectioncited by 26
  25. 2020Gaussian Process Boostingcited by 25
  26. 2024Interpretable Concept Bottlenecks to Align Reinforcement Learning Agentscited by 23
  27. 2023A Comprehensive Survey on Rare Event Predictioncited by 18
  28. 2021Contrastive Reinforcement Learning of Symbolic Reasoning Domainscited by 17
  29. 2020StochasticRank: Global Optimization of Scale-Free Discrete Functionscited by 13
  30. 2021A Boosting Approach to Reinforcement Learningcited by 12
  31. 2016Exploratory Gradient Boosting for Reinforcement Learning in Complex Domainscited by 8
  32. 2022Which Tricks Are Important for Learning to Rank?cited by 3
  33. 2022Boosted Off-Policy Learningcited by 3
  34. 2024Implementing local-explainability in Gradient Boosting Trees: Feature Contributioncited by 2

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Mitigating Information Loss in Tree-Based Reinforcement Learning via Direct Optimizationcited by 5