Improved Algorithm for Independent Component Analysis (ICA) with the Relax and Split Approximation Public

Abstract

Independent component analysis (ICA) has been increasingly used to separate sources and extract features in signal processing and neuroimaging studies. To overcome its computational problems with local optima, as well as problems with non-smooth and non-convex objective functions, relax and split optimization was applied in this study and comparisons were made between the refined algorithms and the popular FastICA algorithm. A tuning parameter was used to control the relaxation and sparsity level of the Relax-Laplace method (with an objective function derived from the Laplace density), and to control the relaxation level of the Relax-logistic method (with an objective function derived from the logistic density).

We conducted a simulation study to examine the impact of the tuning parameter on accuracy and sensitivity to initialization. We found smaller values of the tuning parameter can lead to accurate estimates of the components while having fewer issues with local optima relative to FastICA, while larger values can result in inaccuracies. Running 1000 times with a pool of 50 initializations, we found Relax-Laplace algorithm was the most accurate and consistent one compared with Relax-logistic, FastICA-logistic, and FastICA-tanh.

We conducted a multi-subject analysis of functional magnetic resonance imaging (fMRI) data from the Human Connectom Project using Relax-Laplace, FastICA-logistic, and FastICA-tanh. In a pool of 50 initializations, the Relax-Laplace returns the same result for all initializations, whereas both FastICA-logistic and FastICA-tanh converged on the estimate of the argmax in just over half of the initializations. Moreover, the Relax-Laplace produced sparse figures for the rs-fMRI data that highlight features of resting-state networks.

Table of Contents

1     Introduction 1

2     Method 4

2.1   Classic ICA and Linear Non-Gaussian Component Analysis (LNGCA) 4

2.2   Entropy and Mutual Information 5

2.3   Negentropy and Approximation for FastICA Algorithm 7

2.4   Relax and Split Optimization for ICA 8

2.5   Solution for Laplace Density 10

2.6   Newton’s Method for Continuous Functions 12

2.7   Orthogonal Procrustes Problem 13

2.8   Simulation Study 15

2.9   Real Data Application 16

3     Results 17

3.1   Tuning Scale Parameters 17

3.2   Convergence and Accuracy of the 4 Methods 19

3.3   The First Success Probability of the 4 Methods 20

3.4   Applications on the Real Data 23

4     Discussion 26

Bibliography 28

About this Master's Thesis

Rights statement

• Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.

School	Rollins School of Public Health
Department	Biostatistics
Subfield / Discipline	Biostatistics - MPH & MSPH
Degree	M.S.P.H.
Submission	Master's Thesis
Language	English
Research Field	Biology, Neuroscience Biology, Biostatistics Statistics
Mot-clé	Non-Gaussian component analysis FastICA Sparse ICA Laplace density Relax and split approximation Nonconvex optimization Independent component analysis
Committee Chair / Thesis Advisor	Risk, Benjamin B., Emory University
Committee Members	Chang, Howard, Emory University

Dernière modification

Primary PDF

Thumbnail	Title	Date Uploaded	Actions
	Improved Algorithm for Independent Component Analysis (ICA) with the Relax and Split Approximation ()	2020-04-23 15:53:55 -0400	Press to Select an action Download

Supplemental Files

Thumbnail	Title	Date Uploaded	Actions