Applications of Closed-loop Control in Biomedical Interventions: From Neural Modulation to Diabetes Management Öffentlichkeit

Abstract

Dynamic adaptation of interventions to an individual's real-time medical condition has enormous potential in healthcare. Existing biomedical interventions, such as deep brain stimulation (DBS) for neurodegenerative disorders and insulin therapy for Type 1 diabetes (T1D), typically rely on predetermined treatment plans known as open-loop control. However, these static approaches often overlook the continuous changes in an individual's physiological state. This lack of real-time adaptation can lead to suboptimal treatment outcomes and an inability to manage fluctuating patient conditions effectively.

   

  To address these limitations, we develop a closed-loop control framework that continuously adjusts treatment based on real-time physiological signals. We characterize neuromodulation and blood glucose regulation as control problems, leveraging a combination of machine learning (ML) and optimal control (OC) theory. Our approach integrates neural networks with classic OC techniques like Pontryagin's Maximum Principle and Hamilton-Jacobi-Bellman equations, enabling adaptive control that is robust to physiological variations -- a highly desirable outcome in clinical settings. We utilize established models (Hodgkin-Huxley for neurons, Bergman's Minimal model for glucose-insulin) to simulate and optimize these closed-loop control strategies in a virtual environment, i.e., in silico. Additionally, we define relevant cost functions that quantify clinical objectives to guide the optimization process. We explore various control strategies, including using neural networks to learn optimal treatment adjustments in real-time, overcoming the limitations of open-loop approaches and potentially leading to improved clinical outcomes.

Table of Contents

1 Introduction 1

1.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Research Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2.1 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.3 Dissertation Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Hitchhiker’s Guide to Solving Control Problems 6

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.1.1 Control Formulation . . . . . . . . . . . . . . . . . . . . . . . 7

2.1.2 Ties to Biomedical Applications . . . . . . . . . . . . . . . . . 9

2.2 Closed-loop Control in Biomedical Systems . . . . . . . . . . . . . . . 10

2.2.1 Classical Control Approaches . . . . . . . . . . . . . . . . . . 11

2.2.2 Learning-based Control . . . . . . . . . . . . . . . . . . . . . . 13

2.3 Reinforcement Learning and Optimal Control . . . . . . . . . . . . . 15

2.3.1 Value Functions in RL . . . . . . . . . . . . . . . . . . . . . . 16

2.3.2 Strategies for solving RL problems . . . . . . . . . . . . . . . 18

2.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3 Closed-loop Neuromodulation via Machine Learning and Optimal Control 22

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.2.1 Neuronal Dynamics . . . . . . . . . . . . . . . . . . . . . . . . 24

3.2.2 Neuromodulation Cost Function . . . . . . . . . . . . . . . . . 27

3.3 Model-based Approach to Neuromodulation . . . . . . . . . . . . . . 29

3.3.1 Value Function Approximation with Neural Networks . . . . . 29

3.4 Numerical Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.4.1 Neuromodulatory Effects . . . . . . . . . . . . . . . . . . . . . 31

3.4.2 Optimal Control of Neuronal Dynamics . . . . . . . . . . . . . 32

3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

4 Glucose-Insulin Control 39

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.2 Problem Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

4.2.1 Glucose-Insulin Dynamics . . . . . . . . . . . . . . . . . . . . 41

4.2.2 Cost Function Formulation . . . . . . . . . . . . . . . . . . . . 42

4.3 Classical PID Control . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

4.4 Neural Network-based Glycemic Control . . . . . . . . . . . . . . . . 45

4.4.1 Model-based Control via NeuralHJB . . . . . . . . . . . . . . 45

4.4.2 Data-driven Control . . . . . . . . . . . . . . . . . . . . . . . 46

4.5 Numerical Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4.5.1 Pathological Glucose-Insulin Dynamics . . . . . . . . . . . . . 48

4.5.2 Glycemic Control . . . . . . . . . . . . . . . . . . . . . . . . . 50

4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

5 Conclusion and Research Outlook 55

5.1 Dissertation Summary . . . . . . . . . . . . . . . . . . . . . . . . . . 56

5.2 Research Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

5.2.1 Large-scale Dynamics . . . . . . . . . . . . . . . . . . . . . . . 58

5.2.2 Integrating External Factors . . . . . . . . . . . . . . . . . . . 58

5.2.3 Other Biomedical Applications . . . . . . . . . . . . . . . . . . . . . . . . 59

Bibliography 60

Index 77

About this Dissertation

Rights statement

• Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.

School	Laney Graduate School
Department	Computer Science and Informatics
Degree	Ph.D.
Submission	Dissertation
Language	English
Research Field	Computer Science Applied Mathematics
Stichwort	Type 1 Diabetes Management Machine Learning Glycemic Control Closed-loop Control Reinforcement Learning Value Function Approximation Neuromodulation Optimal Control Neural Networks Deep Brain Stimulation
Committee Chair / Thesis Advisor	Lars Ruthotto, Emory University
Committee Members	Carl Yang, Emory University Nicholas Au Yong, Emory University Razieh Nabi, Emory University

Zuletzt geändert

Primary PDF

Thumbnail	Title	Date Uploaded	Actions
	Applications of Closed-loop Control in Biomedical Interventions: From Neural Modulation to Diabetes Management ()	2024-05-02 19:41:32 -0400	Press to Select an action Download

Supplemental Files

Thumbnail	Title	Date Uploaded	Actions