Sample-Efficient Reinforcement Learning Controller for Deep Brain Stimulation in Parkinson's Disease
Not on the map. Why?
Abstract (arXiv)
Deep brain stimulation (DBS) is an established intervention for Parkinson's disease (PD), but conventional open-loop systems lack adaptability, are energy-inefficient due to continuous stimulation, and provide limited personalization to individual neural dynamics. Adaptive DBS (aDBS) offers a closed-loop alternative, using biomarkers such as beta-band oscillations to dynamically modulate stimulation. While reinforcement learning (RL) holds promise for personalized aDBS control, existing methods suffer from high sample complexity, unstable exploration in binary action spaces, and limited deployability on resource-constrained hardware. We propose SEA-DBS, a sample-efficient actor-critic framework that addresses the core challenges of RL-based adaptive neurostimulation. SEA-DBS integrates a predictive reward model to reduce reliance on real-time feedback and employs Gumbel Softmax-based exploration for stable, differentiable policy updates in binary action spaces. Together, these components improve sample efficiency, exploration robustness, and compatibility with resource-constrained neuromodulatory hardware. We evaluate SEA-DBS on a biologically realistic simulation of Parkinsonian basal ganglia activity, demonstrating faster convergence, stronger suppression of pathological beta-band power, and resilience to post-training FP16 quantization. Our results show that SEA-DBS offers a practical and effective RL-based aDBS framework for real-time, resource-constrained neuromodulation.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 15 of its 43 references that are in the map. First posted on arXiv 8 Jul 2025.
Builds on
15 of 43 references.
References · 15
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
- 2023Mastering Diverse Domains through World Modelscited by 1,193
- 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
- 2019Model-Based Reinforcement Learning for Ataricited by 495
- 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
- 2022Temporal Difference Learning for Model Predictive Controlcited by 375
- 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
- 2021Mastering Atari Games with Limited Datacited by 235
- 2020Actor-Critic Reinforcement Learning for Control with Stability Guaranteecited by 39
- 2024EfficientZero V2: Mastering Discrete and Continuous Control with Limited Datacited by 39
3 more
- 2023Reinforcement Learning with Foundation Priors: Let the Embodied Agent Efficiently Learn on Its Owncited by 16
- 2023Offline Learning of Closed-Loop Deep Brain Stimulation Controllers for Parkinson Disease Treatmentcited by 8
- 2020Reinforcement Learning Framework for Deep Brain Stimulation Studycited by 6
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Temporal Basis Function Models for Closed-Loop Neural Stimulation
- 2022MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrations
- 2025Reinforcement Learning from Implicit Neural Feedback for Human-Aligned Robot Control
- 2020Learning Sparse Rewarded Tasks from Sub-Optimal Demonstrations
- 2020Neural Dynamic Policies for End-to-End Sensorimotor Learning
- 2025Measure gradients, not activations! Enhancing neuronal activity in deep reinforcement learning
- 2023Towards AI-controlled FES-restoration of arm movements: Controlling for progressive muscular fatigue with Gaussian state-space models
- 2026WIMLE: Uncertainty-Aware World Models with IMLE for Sample-Efficient Continuous Control