Multi-task Multi-sensor Framework for Assessing Health Effects of Heat Exposure with Medical Sensors Open Access

Abstract

Medical sensors help to monitor and collect data that make early treatment decisions possible. Using the well-collected data from these sensors, recent studies have focused on the early predictions for some life-threatening conditions, such as dehydration and kidney injuries, in order for early intervention and prevention. In previous studies, the use of multi-sensor fusion transformer models has been proved to be very effective in fusing multi-modal data and detecting signals for various classification tasks. Since many of these tasks are closely related, it is reasonable to incorporate them into a multi-task learning (MTL) framework, so that task-specific information can be shared across different tasks, and that the training and inference cost could be brought down significantly. In this study, we propose a multi-task multi-modal transformer framework that handles several medical predictions. We combine task-specific losses by a dynamic weighting strategy that balances individual losses. To better tackle the label noise problems of our dataset, we also incorporate a teacher-free regularization method (Tf-KD) into our framework. We evaluate the method on the classifications of acute kidney injury (AKI) and dehydration (USG) on Girasoles sensor dataset. We find that MTL benefits both tasks, while Tf-KD only helps the prediction of AKI, suggesting that further research needs to be done.

Table of Contents

1 Introduction 1

1.1 Challenges................................. 2

1.2 Contributions ............................... 3

2 Background 5

2.1 Multi-modal fusion ............................ 5

2.2 Multi-task learning ............................ 6

2.3 Knowledge distillation as regularization . . . . . . . . . . . . . . . . . 7

3 Proposed method 10

3.1 Multi-sensor fusion ............................ 10

3.2 Multi-task learning ............................ 10

3.3 Teacher-free knowledge distillation ................... 12

4 Experiments 14

4.1 Experimental setup............................ 14

4.2 Girasoles sensor dataset ......................... 16

4.3 Multi-task learning ............................ 17

4.4 Teacher-free knowledge distillation (Tf-KD) . . . . . . . . . . . . . . 19

4.5 Multi-task learning and Tf-KD regularization . . . . . . . . . . . . . 20

5 Conclusion 21

6 Future study 22

Bibliography 24 

About this Honors Thesis

Rights statement

• Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.

School	Emory College
Department	Computer Science
Degree	B.S.
Submission	Honors Thesis
Language	English
Research Field	Computer Science Biology, Biostatistics
Keyword	agricultural worker time-series analysis knowledge distillation multi-modality multi-task learning heat-related illness multi-modal data fusion
Committee Chair / Thesis Advisor	Xiong, Li, Emory University
Committee Members	Choi, Jinho, Emory University Hertzberg, Vicki, Emory University

Last modified

Primary PDF

Thumbnail	Title	Date Uploaded	Actions
	Multi-task Multi-sensor Framework for Assessing Health Effects of Heat Exposure with Medical Sensors ()	2023-04-12 12:12:24 -0400	Press to Select an action Download

Supplemental Files

Thumbnail	Title	Date Uploaded	Actions