A Reinforcement Learning Based Decision Support System for Heparin Dosing Open Access

Lin, Rongmei (Fall 2017)

Permanent URL: https://etd.library.emory.edu/concern/etds/x346d4165?locale=en
Published

Abstract

Medication dosing is a comprehensive problem with uncertainties. Every pa- tient has unique condition, meanwhile some drugs have narrow therapeutic windows. Mis-dosing might result in preventable adverse event. Therefore, a robust decision support system would be helpful to clinicians by providing advisable dosing suggestions. Heparin is one of the sensitive drugs. In order to build up the decision support system for heparin patients, we present a clinician in the loop framework with deep reinforcement learning algorithm. There are two main objectives in this thesis, the first one is providing in- dividualized dosing suggestion based on the multi-dimensional features of patients. The second one is evaluating the dosing predicted by our deci- sion support system. We implemented several experiments to achieve these objectives. The data used in the experiments including simulated data, MIMIC-II intensive care unit data and Emory hospital intensive care unit data. There are two important processes with respect to our objectives. In the training process, the decision support system learned from the dosing executed by clinicians and the corresponding response of patients. In the evaluating process, we explored the results from several aspects and focused on the causality between variables and outcomes. The experimental results suggested that given the states of patients, our medication dosing support system is able to provide a reasonable recommendation. 

Table of Contents

1 Introduction 6

1.1 Background ............................ 6

1.2 Approaches ............................ 8

1.2.1 Reinforcementlearning.................. 8

1.2.2 Valuebasedalgorithm .................. 10

1.2.3 Policybasedalgorithm.................. 11

2 Data preprocessing 15

2.1 Simulateddata .......................... 15

2.1.1 Coagulationmodel .................... 15

2.1.2 Simulationprocess .................... 16

2.2 MIMIC-II ............................. 18

2.2.1 MIMIC-IIdatadescription................ 18

2.2.2 MIMIC-IIdatapreprocessing .............. 20

2.3 EmoryICUdata ......................... 21

2.3.1 EmoryICUdatadescription............... 21

2.3.2 EmoryICUdatapreprocessing ............. 23

2.3.3 Emoryheparinprotocol ................. 24

3 Experiment procedures 26

3.1 Preliminary ............................ 26

3.2 Proposedmethod......................... 27

3.3 Experimentaldetails ....................... 30

3.3.1 simulateddataexperiment................ 30

3.3.2 realdataexperiment ................... 32

4 Results and evaluation 35

4.1 Simulateddata .......................... 35

4.1.1 Results........................... 35

4.1.2 Evaluation......................... 38

4.2 MIMIC-II ............................. 40

4.2.1 Results........................... 40

4.2.2 Evaluation......................... 42

4.3 EmoryICUdata ......................... 48

4.3.1 Results........................... 48

4.3.2 Evaluation......................... 53

4.3.3 Classificationtask..................... 58

5 Conclusion 61

Bibliography 63 

About this Master's Thesis

Rights statement
  • Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.
School
Department
Degree
Submission
Language
  • English
Research Field
Keyword
Committee Chair / Thesis Advisor
Committee Members
Last modified

Primary PDF

Supplemental Files