Fast Gaussian Process Solver Público

Abstract

Gaussian Process is a non-parametric, stochastic machine learning technique that has been widely utilized in many fields, such as spatial statistics, geology and computer experiments. However, the application of this powerful technique is limited by its cubic computational complexity. To solve this problem, we studied the intrinsic structures inherent in the kernel matrix and developed linear-complexity solvers to quickly train the Gaussian process model. In this study, we worked on the following aspects:

· Combining stochastic Lanczos Quadrature with density of states, which is a concept developed in quantum physics, to develop a log-determinant solver

· Using the Sherman-Morrison-Woodbury preconditioned conjugate gradient to develop a linear system solver

· Substituting the naive grid search with Bayesian optimization to quickly train the optimal hyper-parameters

After the above steps, we applied it to the synthetic data set and global temperature data to compare our predictions with the true temperature values in order to validate our algorithm. 

Table of Contents

1 Introduction

1.1 The Multivariate Gaussian Distribution

1.2 Gaussian Process

1.3 Motivations

1.4 Kernel Function/Matrix

1.5 Log Marginal Function

1.6 Application of Gaussian Process

1.6.1 Application in Finance/Porfolio

1.6.2 Application in Robotic Techniques

1.6.3 Application in Biosystem

1.6.4 Application in Weather Forecasts

1.7 Difficulties and Contribution

2. Log Determinant Solver

2.1 Matrix Functions

2.2 Stochastic Lanczos Quadrature

2.2.1 Hutchinson's Method

2.2.2 Lanczos Algorithm

2.2.3 Lanczos Quadrature

2.3 Density of States

2.3.1 Limitations of DOS algorithm

2.4 Modified DOS

2.4.1 Truncation of DOS

2.4.2 Comparison

3. Linear System Solver

3.1 Conjugate Gradient(CG) Algorithm

3.1.1 Convergence of CG Algorithm

3.2 Preconditioned Conjugate Gradient

3.3 SMW Preconditioned Conjugate Gradient

3.4 Comparison

4. Hyper-parameter Training

4.1 Naive Grid Search

4.1.1 Synthetic Data Experiment

4.2 Bayesian Optimization

4.3 Comparison

5. Experiment Outline

5.1 Data Description

5.2 Dataset Resize

5.3 Dataset Snapshot

5.4 Result

5.5 Comparison

6. Conclusion

About this Honors Thesis

Rights statement

• Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.

School	Emory College
Department	Mathematics
Degree	B.S.
Submission	Honors Thesis
Language	English
Research Field	Applied Mathematics Computer Science Mathematics
Palabra Clave	Machine Learning Gaussian Process
Committee Chair / Thesis Advisor	Yuanzhe Xi, Emory University
Committee Members	Benjamin Miller, Emory University Bree Ettinger, Emory University Rho Seunghwa, Emory University

Última modificación

Primary PDF

Thumbnail	Title	Date Uploaded	Actions
	Fast Gaussian Process Solver ()	2020-04-23 17:31:33 -0400	Press to Select an action Download

Supplemental Files

Thumbnail	Title	Date Uploaded	Actions