# Coprocessor Actor Critic: A Model-Based Reinforcement Learning Approach For Adaptive Brain Stimulation

arXiv: https://arxiv.org/abs/2406.06714 · HTML: https://mlsanity.com/paper/2406.06714/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 6 of its 40 references that are in the map.
- First posted on arXiv 10 Jun 2024.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 6 of 6)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [When to Trust Your Model: Model-Based Policy Optimization](https://mlsanity.com/paper/1906.08253/) (2019)
- [Offline Learning of Closed-Loop Deep Brain Stimulation Controllers for Parkinson Disease Treatment](https://mlsanity.com/paper/2302.02477/) (2023)
- [Meta-active Learning in Probabilistically-Safe Optimization](https://mlsanity.com/paper/2007.03742/) (2020)
- [Neural Co-Processors for Restoring Brain Function: Results from a Cortical Model of Grasping](https://mlsanity.com/paper/2210.11478/) (2022)

## Built on by (most cited first, 2 of 2)

- [Neurophysiologically Realistic Environment for Comparing Adaptive Deep Brain Stimulation Algorithms in Parkinson Disease](https://mlsanity.com/paper/2505.09624/) (2025)
- [Temporal Basis Function Models for Closed-Loop Neural Stimulation](https://mlsanity.com/paper/2507.15274/) (2025)

## Abstract (arXiv)

Adaptive brain stimulation can treat neurological conditions such as Parkinson's disease and post-stroke motor deficits by influencing abnormal neural activity. Because of patient heterogeneity, each patient requires a unique stimulation policy to achieve optimal neural responses. Model-free reinforcement learning (MFRL) holds promise in learning effective policies for a variety of similar control tasks, but is limited in domains like brain stimulation by a need for numerous costly environment interactions. In this work we introduce Coprocessor Actor Critic, a novel, model-based reinforcement learning (MBRL) approach for learning neural coprocessor policies for brain stimulation. Our key insight is that coprocessor policy learning is a combination of learning how to act optimally in the world and learning how to induce optimal actions in the world through stimulation of an injured brain. We show that our approach overcomes the limitations of traditional MFRL methods in terms of sample efficiency and task success and outperforms baseline MBRL approaches in a neurologically realistic model of an injured brain.