Transfer and Integration of Knowledge for Irregular Spatiotemporal Data Open Access

Gupta, Shrey (Fall 2024)

Permanent URL: https://etd.library.emory.edu/concern/etds/pc289k633?locale=en
Published

Abstract

This dissertation aims to improve the predictive performance of transfer learning models on irregular spatiotemporal data. Modeling irregular spatiotemporal data collected via ground sensors is complex due to spatial and temporal irregularities such as sparse observations and missing temporal points. Transfer learning on such data is much harder as it requires capturing the existing spatial/temporal irregularities, translating their dependencies across domains, as well as managing the distribution shifts during the transfer process. Therefore, this dissertation has three objectives: improve the generalizability of transfer learning models for regression (continuous-valued data), improve transfer across irregular spatiotemporal data sharing similar feature space, and improve transfer across irregular spatiotemporal data with dissimilar feature space.

We believe achieving these objectives will lead to solutions tailored to handle transfer for spatiotemporal data containing high dimensionality, heterogeneity between domains, and spatial/temporal irregularities. An application that motivates the theme of this dissertation is pollution prediction for regions with few and sparsely distributed sensors, as observed in many developing countries. Designing accurate prediction models for these regions is difficult due to existence of topographical, meteorological, geographical, and temporal dependencies. These dependencies have to be accounted for in the current state-of-the-art transfer models. Hence, the algorithmic improvements achieved in this dissertation would serve as a roadmap to apply and improve transfer learning for such domains

Table of Contents

1 Introduction 1

1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Dissertation Statement . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.3 Dissertation Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Background and Related Works 11

2.1 Instance Transfer Learning . . . . . . . . . . . . . . . . . . . . . . . . 15

2.2 Machine Learning for Spatiotemporal Data . . . . . . . . . . . . . . . 17

2.2.1 Spatiotemporal Data . . . . . . . . . . . . . . . . . . . . . . . 17

2.2.2 Prediction Models for Spatiotemporal Data . . . . . . . . . . . 18

2.2.3 Machine Learning for Pollution Prediction . . . . . . . . . . . 19

2.2.4 Machine Learning for PM2.5 Prediction . . . . . . . . . . . . . 21

2.3 Transfer Learning for Spatiotemporal Data . . . . . . . . . . . . . . . 22

2.3.1 Spatial/Temporal Transfer Learning . . . . . . . . . . . . . . . 23

2.3.2 PM2.5 Estimation via Transfer Learning . . . . . . . . . . . . 25

3 Robust & Generalizable Regression Transfer Learning 27

3.1 Rationale: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.2.1 Boosting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.2.2 Variants of Regression Transfer . . . . . . . . . . . . . . . . . 34

3.2.3 Importance Sampling . . . . . . . . . . . . . . . . . . . . . . . 36

3.2.4 Complexity of Distribution . . . . . . . . . . . . . . . . . . . . 37

3.3 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.3.1 Problem Definition: . . . . . . . . . . . . . . . . . . . . . . . . 39

3.3.2 Approach: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

3.3.3 STrAdaBoost.R2 . . . . . . . . . . . . . . . . . . . . . . . . . 41

3.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

3.4.1 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

3.4.2 Ablation Study . . . . . . . . . . . . . . . . . . . . . . . . . . 47

3.4.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

3.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

4 Transfer across regions w/ similar feature space 54

4.1 Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

4.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

4.3 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

4.4 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

4.4.1 Neighborhood Cloud Generation . . . . . . . . . . . . . . . . . 59

4.4.2 Generating Latent Dependency Factor (LDF) . . . . . . . . . 60

4.4.3 Transfer Learning and Multivariate Regression . . . . . . . . . 62

4.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

4.5.1 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

4.5.2 Prediction Models . . . . . . . . . . . . . . . . . . . . . . . . . 64

4.5.3 Optimal k for Neighborhood Cloud . . . . . . . . . . . . . . . 69

4.5.4 Results and Analysis . . . . . . . . . . . . . . . . . . . . . . . 69

4.5.5 Qualitative Analysis . . . . . . . . . . . . . . . . . . . . . . . 71

4.5.6 Ablation Study . . . . . . . . . . . . . . . . . . . . . . . . . . 72

4.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

4.6.1 Limitations and Future Work . . . . . . . . . . . . . . . . . . 73

4.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

5 Transfer across regions w/ dissimilar feature space 76

5.1 Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

5.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

5.3 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

5.3.1 Neighborhood Cloud Generation (Input Dataset) . . . . . . . 80

5.3.2 Generating LDF via Two-stage Autoencoder Model . . . . . . 80

5.3.3 Regression Transfer Learning . . . . . . . . . . . . . . . . . . 81

5.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

5.4.1 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

5.4.2 Prediction Models . . . . . . . . . . . . . . . . . . . . . . . . . 83

5.4.3 Optimal k for Neighborhood Cloud . . . . . . . . . . . . . . . 84

5.4.4 Feature Standardization . . . . . . . . . . . . . . . . . . . . . 84

5.4.5 Results and Analysis . . . . . . . . . . . . . . . . . . . . . . . 85

5.5 Discussion, Limitations and Future Work . . . . . . . . . . . . . . . . 88

5.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

6 Future Work 92

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

6.2 MSDA via Weighted Combination . . . . . . . . . . . . . . . . . . . . 94

6.2.1 Learning Optimal Weights for Distributed Weighing . . . . . . 95

6.3 MSDA: Two-Stage Domain Adaptation . . . . . . . . . . . . . . . . . 95

6.3.1 Stage 1: Marginal Probability Alignment . . . . . . . . . . . . 96

6.3.2 Stage 2: Conditional Probability Alignment . . . . . . . . . . 96

6.4 MSDA: Adversarial Learning . . . . . . . . . . . . . . . . . . . . . . . 97

6.5 MSDA w/ Sparse Variational GP . . . . . . . . . . . . . . . . . . . . 98

6.5.1 Sparse Variational Gaussian Processes (SVGPs) . . . . . . . . 98

6.5.2 MSDA w/ SVGP . . . . . . . . . . . . . . . . . . . . . . . . . 100

6.6 MSDA and Heterogeneous Data . . . . . . . . . . . . . . . . . . . . . 100

6.6.1 Heterogeneous Data . . . . . . . . . . . . . . . . . . . . . . . 100

6.6.2 MSDA w/ Heterogeneous Data . . . . . . . . . . . . . . . . . 101

6.7 Spatial Indexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

6.7.1 Spatial Indexing for Spatial Transfer Learning . . . . . . . . . 103

7 Conclusion 105

Bibliography 107

About this Dissertation

Rights statement
  • Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.
School
Department
Degree
Submission
Language
  • English
Research Field
Keyword
Committee Chair / Thesis Advisor
Committee Members
Last modified

Primary PDF

Supplemental Files