# Learning Control Policies of Hodgkin-Huxley Neuronal Dynamics

arXiv: https://arxiv.org/abs/2311.07563 · HTML: https://mlsanity.com/paper/2311.07563/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 52 references that are in the map.
- First posted on arXiv 13 Nov 2023.

## Builds on (most cited first, 9 of 9)

- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [A Tour of Reinforcement Learning: The View from Continuous Control](https://mlsanity.com/paper/1806.09460/) (2018)
- [Deep Reinforcement Learning and its Neuroscientific Implications](https://mlsanity.com/paper/2007.03750/) (2020)
- [A Neural Network Approach Applied to Multi-Agent Optimal Control](https://mlsanity.com/paper/2011.04757/) (2020)
- [A Neural Network Approach for Stochastic Optimal Control](https://mlsanity.com/paper/2209.13104/) (2022)
- [Learning Optimal Control via Forward and Backward Stochastic Differential Equations](https://mlsanity.com/paper/1509.02195/) (2015)
- [Semiglobal optimal feedback stabilization of autonomous systems via deep neural network approximation](https://mlsanity.com/paper/2002.08625/) (2020)
- [Reinforcement Learning Framework for Deep Brain Stimulation Study](https://mlsanity.com/paper/2002.10948/) (2020)
- [Mean-field modeling of the basal ganglia-thalamocortical system. I. Firing rates in healthy and parkinsonian states](https://mlsanity.com/paper/1801.01822/) (2018)

## Abstract (arXiv)

We present a neural network approach for closed-loop deep brain stimulation (DBS). We cast the problem of finding an optimal neurostimulation strategy as a control problem. In this setting, control policies aim to optimize therapeutic outcomes by tailoring the parameters of a DBS system, typically via electrical stimulation, in real time based on the patient's ongoing neuronal activity. We approximate the value function offline using a neural network to enable generating controls (stimuli) in real time via the feedback form. The neuronal activity is characterized by a nonlinear, stiff system of differential equations as dictated by the Hodgkin-Huxley model. Our training process leverages the relationship between Pontryagin's maximum principle and Hamilton-Jacobi-Bellman equations to update the value function estimates simultaneously. Our numerical experiments illustrate the accuracy of our approach for out-of-distribution samples and the robustness to moderate shocks and disturbances in the system.