On Using Elemental and Non-Elemental Sets to Reproduce the OLS Estimator in Linear Regression Open Access

Abstract

It has been shown previously that Ordinary Least Squares (OLS) estimates based on a multiple linear regression model with p unknown parameters can be reproduced by combining the results from fitting the same model to all elemental sets, the unique subsets of size p from the total of n observations. In addition, it has been shown that the same goal of reproducing OLS estimates can be achieved by combining the results of the regressions on all unique non-elemental sets, i.e., subsets of size k where p + 1 ≤ k ≤ n − 1. We consider three new methods aimed at reproducing the overall OLS estimates of parameters. The three methods use the direct inverse-variance(INV) weights, the refined inverse-variance (REF) and the constrained optimal (CON) weights applied to each individual OLS estimator based on elemental or non-elemental sets. These methods are compared with the determinant-based weighting method which has previously been proven to reproduce the overall OLS estimates. The primary new insight gained by our study is the notion that the direct inverse-variance weighting essentially achieves the objective, while in theory there may be an infinitely large collection of different weights that can do so. We illustrate the use of the various weighting schemes using simulated data under various linear regression settings, including one-way and two-way ANOVA designs.

Table of Contents

Contents

1 Introduction 1
1.1 Elemental Sets and Elemental Regressions 1
1.2 Existing Estimators as Functions of Elemental Regressions by OLS 2
1.3 Outline 3
2 Methods 6
2.1 Methods to Reproduce the OLS Estimator in Linear Regression 6
2.1.1 The Direct Inverse-Variance Weighted Average Estimator 6
2.1.2 The Refined Inverse-Variance Weighted Average Estimator 7
2.1.3 The Constrained Optimal Weighted Average Estimator 11

3 Simulation Studies 13
3.1 Special Case 1: Simple linear regression (SLR) with n=3, p=2 13
3.2 Special Case 2: Multiple linear regression (MLR) with n=7, p=4 17
3.2.1 Reproducing OLS coefficients via the elemental sets with n=7, k=4, p=4 18
3.2.2 Reproducing OLS coefficients via the non-elemental sets with n=7, k=5, p=4
21
3.2.3 Reproduce the coefficients by the non-elemental sets with n=7, k=6, p=4
23
3.3 Evaluation of direct INV method as n increases 26
3.4 Special Case 3: Reproducing OLS coefficients for one-way ANOVA 28
3.4.1 Reproducing the coefficients of the balanced one-way ANOVA 29
3.4.2 Reproducing the coefficients of the unbalanced one-way ANOVA 32
3.5 Special Case 4: Reproducing the coefficients of two-way ANOVA 33
3.5.1 Reproducing the coefficients of the balanced two-way ANOVA 34
3.5.2 Reproducing the coefficients of the unbalanced two-way ANOVA 38
4 SUMMARY AND FUTURE WORK 40
4.1 Summary 40
4.2 Future Research 41

About this Master's Thesis

Rights statement

• Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.

School	Rollins School of Public Health
Department	Biostatistics and Bioinformatics
Subfield / Discipline	Biostatistics - MPH & MSPH
Degree	MPH
Submission	Master's Thesis
Language	English
Research Field	Health Sciences, Public Health
Keyword	reproduce the OLS estimators elemental sets non-elemental sets
Committee Chair / Thesis Advisor	Lyles, Robert, Emory University
Partnering Agencies	Does not apply (no collaborating organization)

Last modified

Primary PDF

Thumbnail	Title	Date Uploaded	Actions
	On Using Elemental and Non-Elemental Sets to Reproduce the OLS Estimator in Linear Regression ()	2018-08-28 14:05:34 -0400	Press to Select an action Download

Supplemental Files

Thumbnail	Title	Date Uploaded	Actions