Precision Neuromodulation Therapies Using Artificial Intelligence Restricted; Files Only

Abstract

Implantable neuromodulation devices, such as deep brain stimulation (DBS) and vagus nerve stimulation (VNS) have revolutionized neuroscience research and clinical care thanks to their ability to directly intervene in pathological circuits. These implantable devices provide a powerful paradigm for treating neurological disorders, restoring and enhancing neural functions, and understanding the causal links between neural and behavioral processes. Yet, despite their growing adoption in clinical care, challenges persist, impeding their seamless integration into standard of care. Recent advancements in next-generation implantable devices offer considerable customization in stimulation parameters, paving the way for delivering precision neuromodulation therapies. Moreover, given the variations in electrode placement, local anatomy, and the diversity in symptom type and severity, it is imperative to design adaptive, patient-specific treatments. Proper programming of implantable devices is a critical step for optimizing patients' therapeutic outcomes and avoiding inducing adverse side effects. Despite the efforts in developing standard clinical guidelines for programming neuromodulation devices, these approaches are very time-consuming and may lead to sub-optimal therapy for patients. Additionally, these approaches do not take into account the complex and dynamic nature of the nervous system, which can change over time and require ongoing adjustment of stimulation parameters. Therefore, there is a growing need to develop automated intelligent closed-loop neuromodulation systems (iCLON) to facilitate the programming of implantable devices.

Despite these challenges, the potential benefits of these systems for treating neurological disorders make this a promising area of research and development. Designing automated closed-loop neuromodulation systems is a complex task and requires multifaceted considerations. This research is an effort toward facilitating the design and development of automated iCLON systems by developing a translational design paradigm. This dissertation contributed to the development of multiple simulation environments, pivotal in the design of novel and effective iCLON systems. The simulation platforms offer a safe and controlled environment for rigorous testing and refinement before clinical implementation. This research also introduces multiple control tasks, replicating the actual experimental and clinical applications and ensuring reproducibility and easier translation from simulation to clinical practice. Moreover, a control policy is at the core of iCLON systems which automatically learns and adjusts the stimulation parameters. In this research, I developed data-driven control strategies using optimization and reinforcement learning techniques that are able to learn and optimize neuromodulation control strategies autonomously, via closed-loop interaction with the nervous system. Notably, I developed and clinically evaluated a fully automated DBS programming framework for treatment of tremor in patients with Parkinson’s disease and essential tremor that was shown to be efficient and safe while providing outcomes comparable to that achieved by expert clinicians. Finally, this research presents a collaborative effort towards developing an end-to-end translational platform for the design and implementation of iCLON systems. Utilizing an algorithm-hardware co-design approach, the platform facilitates the exploration of brain-implantable devices capable of autonomously learning and adapting control policies. This platform aims to enable research and development of brain-implantable iCLON systems for a wide community of neuroscientists, clinicians, and engineers.

Table of Contents

1       Introduction 1 

1.1   Motivation 1 

1.2   Challenges and requirements of designing intelligent closed-loop neuromodulation systems 2 

1.3   Significance and contributions 5 

1.4   Thesis outline 9 

1.5   List of publications 10 

2       A review on closed-loop neuromodulation systems, with a focus on control policy algorithms 13 

2.1   Introduction 13 

2.2   Introduction to closed-loop neuromodulation control 16 

2.3   Control strategies 18 

2.3.1      Open-loop control 18 

2.3.2      Closed-loop control 19 

2.3.3      Adaptive control 20 

2.3.4      Model-based control 20 

2.3.5      Model-free control 21 

2.4   Classical control algorithms in iCLON systems 21 

2.4.1      On-Off and threshold-based controller 22 

2.4.2      Proportional-integral-derivative control 23 

2.4.3      Delayed-feedback controller 24 

2.4.4      Fuzzy logic controller 24 

2.4.5      Model predictive control 25 

2.5   Recent trends in developing novel closed-loop neuromodulation systems 26 

2.6   The need for developing research platforms to enable research and development of implantable iCLON systems 29 

3       Automated deep brain stimulation programming with safety constraints for tremor suppression in patients with Parkinson’s disease and essential tremor 31 

3.1   Introduction 31 

3.2   Patient selection criteria and clinical experiment procedure 34 

3.3   Automated DBS programming framework: software design 37 

3.4   GPR modelling of the effect of DBS settings using a quantified objective measure 39 

3.5   DBS programming algorithms 43 

3.5.1      Bayesian optimization 43 

3.5.2      Safe Bayesian optimization 46 

3.6   Stopping criteria and advanced optimization 51 

3.7   Results 53 

3.7.1      Quantifying tremor response to stimulation 53 

3.7.2      Comparison of the clinical settings and the automated settings 55 

3.7.3      Speed of convergence of the automated DBS programming System 56 

3.8   Discussion 59 

3.9   Conclusion 64 

4       Reinforcement learning for closed-loop regulation of cardiovascular system with selective vagus nerve stimulation 66 

4.1   Introduction 66 

4.2   Simulation environments 72 

4.2.1      Standard API for rat cardiac model 72 

4.2.2      In-silico rat cardiac model 73 

4.2.3      Reduced order model of the physiological rat cardiac model using temporal convolutional neural networks 74 

4.3   Experimental design 75 

4.3.1      Regulating cardiovascular system using RL through designing a set point tracking task 75

4.3.2      Designing a general policy using deep RL algorithms 76

4.3.3      Designing an adaptive policy using PILCO 77

4.4   Reinforcement learning agents 78 

4.4.1      Proximal policy optimization algorithm 79 

4.4.2      Soft actor-critic algorithm 80 

4.4.3      Probabilistic inference for learning and control 80 

4.4.4      Reward function 82 

4.5   Results 82 

4.5.1      Performance of TCN model 82 

4.5.2      Training performance of RL agents 83 

4.5.3      Performance of deep RL agents in set-point tracking task in four cardiac models 84 

4.5.4      Performance of PILCO in set-point tracking task in four cardiac models 84 

4.5.5      Adaptability of PILCO to variations in target set point 85 

4.5.6      Adaptability of PILCO to variations in the underlying dynamics of the environment 86 

4.5.7      Adaptability of deep RL agents to variations in the underlying dynamics of the environment using transfer learning 87 

4.6   Discussion 88 

4.7   Conclusion 91 

5       Neuroweaver: a translational platform for embedding artificial in- telligence in closed-loop neuromodulation systems 93 

5.1   Introduction 93 

5.2   Challenges and considerations 97 

5.3   Neuroweaver in a glance 100 

5.4   Neuroweaver platform 102 

5.4.1      Cross-domain programming interface in python 102 

5.4.2      Multi-target cross-domain compilation 104 

5.5   An example implementation with CNF program using the CDI in Python 105 

5.6   Simulation environments and control tasks for designing iCLON systems 109 

5.6.1      Interactive AI-enabled closed-loop synchrony suppression in Bon- hoeffer–van der Pol model 109 

5.7   RL algorithms integrated in the design of iCLON systems 111 

5.7.1      Proximal policy optimization 113 

5.7.2      Soft actor-critic network 114 

5.7.3      Deep deterministic policy gradient 115 

5.7.4      Model-based reinforcement learning with model predictive control 115 

5.7.5      Probabilistic inference for learning control 116 

5.8   Results 117 

5.8.1      Synchrony suppression using reinforcement learning algorithms 117

5.8.2      CNF implementation of iCLON systems using deep RL algorithms 119

5.8.3      FPGA execution of deep RL agents in inference mode 120 

5.8.4      In-vivo experiments 122 

5.9   Discussion 123 

5.10                Conclusion 125 

6       Conclusion and future direction 127 

6.1   Contributions to the field 127 

6.2   Future work 129 

7       Bibliography 131 

About this Dissertation

Rights statement

• Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.

School	Laney Graduate School
Department	Computer Science and Informatics
Degree	Ph.D.
Submission	Dissertation
Language	English
Research Field	Computer Science
Keyword	Vagus Nerve Stimulation Reinforcement learning Artificial Intelligence Optimization Closed-loop Neuromodulation Neurological Disorders Machine Learning Deep Brain Stimulation
Committee Chair / Thesis Advisor	Babak Mahmoudi, Emory University
Committee Members	Rishi Kamaleswaran, Emory University Mayuresh Kothare, Lehigh University Johnathan McKay, Emory University

Last modified

Primary PDF

Thumbnail	Title	Date Uploaded	Actions
	File download under embargo until 10 January 2026	2023-12-12 20:43:57 -0500	File download under embargo until 10 January 2026

Supplemental Files

Thumbnail	Title	Date Uploaded	Actions