Pre-Training Graph Neural Networks for Data-Efficient Brain Network Analysis Public

Abstract

The human brain is the central hub of the neurobiological system, controlling behavior and cognition in complex ways. Recent advances in neuroscience and neuroimaging analysis have shown a growing interest in the interactions between brain regions of interest (ROIs) and their impact on neural development and disorder diagnosis. As a powerful deep model for analyzing structural data, Graph Neural Networks (GNNs) have been applied for brain network analysis. However, effective training of deep models requires large amounts of labeled data, which is often scarce in brain network datasets due to the complexities of data acquisition and sharing restrictions. To make the most out of available training data, this work examines data- and label-efficient training of GNN model. In particular, the goal is to pre-train GNN to capture intrinsic brain network structures, regardless of clinical outcomes, and is easily adaptable to various downstream tasks. To this end, the proposed framework comprises three key components: (1) a meta-learning based multi-task pre-training platform with dynamic task adaptive reweighing consideration that learns a generalizable model initialization with efficient optimization schedule (2) an unsupervised pre-training objective designed specifically for brain networks, which enables learning from large-scale datasets without task-specific labels; (3) a data-driven atlas mapping pipeline with variance-based ROI alignment mechanism that facilitates knowledge transfer across datasets with different ROI systems. Extensive empirical evaluations using various GNN backbones have demonstrated the robust and superior performance of the proposed framework compared to baseline methods.

Table of Contents

Contents

1 Introduction 1

1.1 BackgroundandMotivation ....................... 1

1.2 OverviewofProposedSolutions ..................... 3

1.3 SummaryofContribution ........................ 4

2 Related Work 6

2.1 GNNsforBrainNetworkAnalysis. ................... 6

2.2 Meta-LearningforGraphClassification................. 7

2.3 Unsupervised Graph Representation Learning and GNN Pre-training. 7

3 Problem Definition 9

4 Data-Efficient Training Strategies 11

4.1 Method 1: Learning Without Pre-training (NPT) . . . . . . . . . . . 12

4.2 Method2: Single-taskTransferLearning(STT) . . . . . . . . . . . . 12

4.3 Method3: Multi-taskTransferLearning(MTT) . . . . . . . . . . . . 13

4.4 Method4: Multi-taskMeta-Learning(MML). . . . . . . . . . . . . . 14

5 Unsupervised Brain Network Pre-training 17

6 Brain Network Oriented Design Considerations 21

6.1 Data-drivenBrainAtlasMapping.................... 21

6.1.1 Challenges............................. 21

6.1.2 Autoencoder with Customized Regularizers . . . . . . . . . . . 22

6.1.3 Variance-basedDimensionSorting. . . . . . . . . . . . . . . . 24

6.2 SourceTaskReweighing ......................... 25

6.2.1 Challenges............................. 25

6.2.2 DynamicTaskReweighing.................... 26

7 Dataset and Experimental Configuration 29

7.1 DatasetDetails .............................. 29

7.1.1 Parkinson’s Progression Markers Initiative (PPMI) . . . . . . 30

7.1.2 BipolarDisorders(BP)...................... 30

7.1.3 Human Immunodeficiency Virus Infection (HIV) . . . . . . . . 31

7.2 ExperimentalSetup............................ 31

7.2.1 Backbone Selection and Evaluation Metric . . . . . . . . . . . 31

7.2.2 GNNSetup ............................ 32

7.2.3 Pre-trainingPipelineSetup ................... 32

7.2.4 AtlasMappingRegularizerSetup ................ 32

7.2.5 DownstreamEvaluationSetup.................. 33

8 Experiments and Analysis 34

8.1 OverallPerformanceComparison(RQ1) ................ 34

8.2 AblationStudies(RQ2) ......................... 38

8.3 Analysis of Two-level Contrastive Sampling (RQ3) . . . . . . . . . . . 40

8.4 AnalysisofROIAlignment(RQ4).................... 41

9 Conclusions and Future Directions 43

Appendix A Autoencoder Structure Analysis 45

A.1 Bridging Reconstruction Minimization and Variance Maximization . . 45

A.2 Variance-basedSortingProcedure.................... 46

Appendix B Additional Experiments 48

B.1 PerformancewithGATandGIN .................... 48

B.2 AdditionalAblationStudiesonDTI................... 48

Bibliography 50 

About this Honors Thesis

Rights statement

• Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.

School	Emory College
Department	Computer Science
Degree	B.S.
Submission	Honors Thesis
Language	English
Research Field	Computer Science
Mot-clé	Data Mining Machine Learning
Committee Chair / Thesis Advisor	Carl Yang, Emory University
Committee Members	Li Xiong, Emory University Lars Ruthotto, Emory University

Dernière modification

Primary PDF

Thumbnail	Title	Date Uploaded	Actions
	Pre-Training Graph Neural Networks for Data-Efficient Brain Network Analysis ()	2023-04-05 14:06:08 -0400	Press to Select an action Download

Supplemental Files

Thumbnail	Title	Date Uploaded	Actions