Penalized Linear Regression and a Flexible Alternative Público

Abstract

We provide a comprehensive review of traditional and modern approaches to estimation and model selection in the linear regression framework. We are particularly interested in methods that estimate regression coefficients under various constraints, and the impact these constraints have on the resulting coefficient estimates and models selected. We propose a novel approach that allows for a more flexible penalty structure, and provide an estimation algorithm that utilizes linear programming. Finally, our flexible estimator is illustrated in various applications that exhibit spatial structure.

Table of Contents

1. Traditional and Modern Approaches to Estimation and Model Selection in Linear Regression
   1. Introduction
   2. Notation
   3. Linear Regression (ordinary least squares)
      1. Definition
      2. Estimation
      3. Variable Selection
      4. Shortcomings
   4. Penalized Linear Regression
      1. Definition
      2. Ridge Regression (L2)
      3. Lasso (L1)
         1. Motivation
         2. Definition
         3. Computation
         4. Theoretical Details
         5. Choosing Lambda
         6. Drawbacks
      4. L1 Extensions
         1. Adaptive Lasso
         2. Elastic Net
         3. Group Lasso
         4. Fused Lasso
      5. Lasso and Generalized Linear Models
   5. Results
      1. Illustrations
      2. Simulations
         1. Description of Scenarios
         2. Simulation 1
         3. Simulation 2
   6. Discussion
2. A Flexible Dantzig Selector
   1. Introduction
   2. Background
      1. Generalized Lasso
         1. Definition
         2. Estimation
         3. Incorporating a Ridge Penalty (L2)
         4. Penalty Matrix, M
         5. Shortcomings
         6. Discussion
      2. Dantzig Selector
         1. Definition
         2. Properties
         3. Computation
         4. Comparisions with Lasso
         5. Extensions
   3. Methods
      1. Flexible Dantzig Selector
         1. Motivation
         2. Definition
         3. Flexible Penalty Matrix (M)
         4. Computation
         5. Speed Tests
         6. Variant 1: Proportional Weighting
         7. Variant 2: Adaptive Weighting
         8. Variant 3: Weighting Observations
         9. Bootstrap-Enhanced Estimation
         10. Randomized Estimation
   4. Results
      1. Illustrations
      2. Simulations
         1. Description of Scenarios
         2. Simulation - Goal 1
         3. Simulation - Goal 2
      3. Data Analysis
         1. Spatially-Informed Test Statistic Thresholding
         2. The Alzheimer's Disease Neuroimaging Initiative (ADNI)
   5. Discussion
3. Conclusion

About this Master's Thesis

Rights statement

• Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.

School	Laney Graduate School
Department	Biostatistics
Degree	MS
Submission	Master's Thesis
Language	English
Research Field	Biology, Biostatistics Statistics
Palabra Clave	linear regression penalized regression dantzig selector
Committee Chair / Thesis Advisor	Waller, Lance, Emory University
Committee Members	Kelley, Mary, Emory University Glanz, Hunter, California Polytechnic State University Hertzberg, Vicki S, Emory University

Última modificación

Primary PDF

Thumbnail	Title	Date Uploaded	Actions
	Penalized Linear Regression and a Flexible Alternative ()	2018-08-28 12:48:30 -0400	Press to Select an action Download

Supplemental Files

Thumbnail	Title	Date Uploaded	Actions