# Reinforcement learning in large, structured action spaces: A simulation study of decision support for spinal cord injury rehabilitation

arXiv: https://arxiv.org/abs/2310.14976 · HTML: https://mlsanity.com/paper/2310.14976/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 154 references that are in the map.
- First posted on arXiv 23 Oct 2023.

## Builds on (most cited first, 20 of 20)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Reinforcement Learning in Healthcare: A Survey](https://mlsanity.com/paper/1908.08796/) (2019)
- [Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendation](https://mlsanity.com/paper/1807.01473/) (2018)
- [Deep Reinforcement Learning for Sepsis Treatment](https://mlsanity.com/paper/1711.09602/) (2017)
- [Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approach](https://mlsanity.com/paper/1705.08422/) (2017)
- [Estimating Dynamic Treatment Regimes in Mobile Health Using V-learning](https://mlsanity.com/paper/1611.03531/) (2016)
- [A Reinforcement Learning Approach to Weaning of Mechanical Ventilation in Intensive Care Units](https://mlsanity.com/paper/1704.06300/) (2017)
- [Improving Sepsis Treatment Strategies by Combining Deep and Kernel-Based Reinforcement Learning](https://mlsanity.com/paper/1901.04670/) (2019)
- [Robust and Efficient Transfer Learning with Hidden-Parameter Markov Decision Processes](https://mlsanity.com/paper/1706.06544/) (2017)
- [Representation and Reinforcement Learning for Personalized Glycemic Control in Septic Patients](https://mlsanity.com/paper/1712.00654/) (2017)
- [Clustering - What Both Theoreticians and Practitioners are Doing Wrong](https://mlsanity.com/paper/1805.08838/) (2018)
- [A dual mode adaptive basal-bolus advisor based on reinforcement learning](https://mlsanity.com/paper/1901.01816/) (2019)
- [An Optimal Policy for Patient Laboratory Tests in Intensive Care Units](https://mlsanity.com/paper/1808.04679/) (2018)
- [Optimization of anemia treatment in hemodialysis patients via reinforcement learning](https://mlsanity.com/paper/1509.03977/) (2015)
- [The Actor Search Tree Critic (ASTC) for Off-Policy POMDP Learning in Medical Decision Making](https://mlsanity.com/paper/1805.11548/) (2018)
- [A Batch, Off-Policy, Actor-Critic Algorithm for Optimizing the Average Reward](https://mlsanity.com/paper/1607.05047/) (2016)
- [Precision medicine as a control problem: Using simulation and deep reinforcement learning to discover adaptive, personalized multi-cytokine therapy for sepsis](https://mlsanity.com/paper/1802.10440/) (2018)
- [Personalized Medical Treatments Using Novel Reinforcement Learning Algorithms](https://mlsanity.com/paper/1406.3922/) (2014)
- [Bounded Optimal Exploration in MDP](https://mlsanity.com/paper/1604.01350/) (2016)
- [Dynamic Measurement Scheduling for Adverse Event Forecasting using Deep RL](https://mlsanity.com/paper/1812.00268/) (2018)

## Abstract (arXiv)

Reinforcement learning (RL) has helped improve decision-making in several applications. However, applying traditional RL is challenging in some applications, such as rehabilitation of people with a spinal cord injury (SCI). Among other factors, using RL in this domain is difficult because there are many possible treatments (i.e., large action space) and few patients (i.e., limited training data). Treatments for SCIs have natural groupings, so we propose two approaches to grouping treatments so that an RL agent can learn effectively from limited data. One relies on domain knowledge of SCI rehabilitation and the other learns similarities among treatments using an embedding technique. We then use Fitted Q Iteration to train an agent that learns optimal treatments. Through a simulation study designed to reflect the properties of SCI rehabilitation, we find that both methods can help improve the treatment decisions of physiotherapists, but the approach based on domain knowledge offers better performance. Our findings provide a "proof of concept" that RL can be used to help improve the treatment of those with an SCI and indicates that continued efforts to gather data and apply RL to this domain are worthwhile.