Differential Person Functioning Público

Abstract

The accuracy and meaningfulness of test scores is a crucial issue in educational settings marked by high-stakes assessments within No Child Left Behind and Race to the Top. Differential person functioning (DPF) is presented in this study using a Rasch measurement framework as a means for assessing the accuracy and validity of scores on educational assessments. The purpose of this study is to further our current understanding of DPF as not only a threat to test score validity, but as a way to examine individual student response behaviors. Erasure analyses and multilevel modeling are used as the methods to identify and assess DPF across various contexts. The following questions are used to guide the research:

(1) What is differential person functioning?
(2) How do the methods for assessing differential person functioning differ across contexts?
(3) To what extent does differential person functioning contribute to our understanding of person fit across contexts?

The first question is answered through an extensive review of literature on the various components of DPF: person measurement, person response functions, person fit indices, and response behaviors. Guiding questions (2) and (3) are explored using data from a high-stakes third grade statewide assessment of mathematics and reading achievement. These questions are explored using two case studies, each replicated within two content areas (mathematics and reading) yielding a total of four contexts that are explored. The first case study investigates the relationship between wrong-to-right erasures, person fit indices, and school-level mathematics and reading achievement using the Many Facets Rasch model (MFRM) and a pre/post erasure design. The second case study uses hierarchical generalized linear modeling (HGLM) to examine student and school factors that may be associated with the aberrant responses of students that include proficiency levels, economic status, gender, and erasure behavior.

The dissertation sheds light on the importance of evaluating DPF when considering the validity evidence for an assessment. Additionally, MFRM and HGLM, yielded valuable information for researchers to begin to consider systematic routine analyses of DPF for high stakes assessments.

Table of Contents

CHAPTER ONE: INTRODUCTION 1
Theoretical Framework 3
Statement of the Problem 8
Purpose of the Study 9
Guiding Questions 10
Definitions 10
Organization of Dissertation 12

CHAPTER TWO: REVIEW OF LITERATURE 13
Reliability of Person Measurements 13
Person Response Functions 15
Person Fit Indices 18
Response Behaviors 24

CHAPTER THREE: CASE STUDY ONE 28
Purpose 29
Research Questions 30
Methods 30
Results 35
Discussion 38

CHAPTER FOUR: CASE STUDY TWO 41
Purpose 44
Research Questions 45
Methods 45
Results 50
Discussion 54

CHAPTER FIVE: DISCUSSION & SUMMARY 56
Guiding Question #1: Differential Person Functioning 57
Guiding Question #2: The Role of Context 59
Guiding Question #3: Person Fit Across Contexts 62
Limitations 63
Implications for Research, Policy, and Practice 64

References 69

Appendix A: IRB Determination Letter 129

List of Tables
Table 1. Chronological List of Key Ideas in Person Measurement 78
Table 2. Abbreviated List of Person Fit Indices 84
Table 3. Summary of Case Studies 87
Table 4. Student Demographics for Grade 3 students in Case Study 88
Table 5. Student Erasures by School (Mathematics Content Area) 89
Table 6. Student Erasures by School (Reading Content Area) 91
Table 7. Facets Summary Statistics (Mathematics Content Area) 93
Table 8. Facets Summary Statistics (Reading Content Area) 94
Table 9. Fit Statistics for Pre and Post Erasure by School (Mathematics Content Area) 95
Table 10. Fit Statistics for Pre and Post Erasure by School (Reading Content Area) 97
Table 11. Mapping of Research Questions and Analysis 99
Table 12. Distribution of Aberrance 100
Table 13. Explanatory variables defined 101
Table 14. Summary Statistics from Facets Analyses 102
Table 15. Means and Standard Deviations 103
Table 16. Percentage of misfitting persons with Outfit MSE or Infit MSE above 1.20 104
Table 17. Parameter Estimates for the Outfit Two-Level Model for Mathematics Content Area 105
Table 18. Parameter Estimates for the Infit Two-Level Model for Mathematics Content Area 106
Table 19. Parameter Estimates for the Outfit Two-Level Model for Reading Content Area 107
Table 20. Parameter Estimates for the Infit Two-Level Model for Reading Content Area 108

List of Figures
Figure 1. Impact of Crossing Person Response Functions 109
Figure 2. Illustration of the Creation of Pre-Erasure Strings 110
Figure 3. Conceptual Models 111
Figure 4. Variable Map for Students and Items (Mathematics Content Area) 112
Figure 5. Variable Map for School, Pre/Post Indicator, Item 113
Figure 6. Variable Map for Students and Items (Reading Content Area) 114
Figure 7. Variable Map for School, Pre/Post Indicator, Items 115
Figure 8. Mean Wrong to Right by Total Erasures at the School Level (Mathematics Content Area) 116
Figure 9. Mean Post Erasure School Mathematics Achievement by Mean Pre Erasure School Mathematics Achievement 117
Figure 10. Pre/Post Erasure Indicator by School Mathematics Achievement 118
Figure 11. Pre and Post Erasure Outfit Z by Mean Wrong to Right at the School Level (Mathematics Content Area) 119
Figure 12. Pre and Post Erasure Z by Mean Wrong to Right at the School Level (Mathematics Content Area) 120
Figure 13. Post Erasure Z by Pre Erasure Z at the School Level (Mathematics Content Area) 121
Figure 14. Mean Wrong to Right by Total Erasures (Reading Content Area) 122
Figure 15. Mean Post Erasure School Reading Achievement by Mean Pre Erasure School Reading Achievement 123
Figure 16. Pre/Post Erasure Indicator by School (Reading Content Area) 124
Figure 17. Pre and Post Erasure Outfit Z by Mean Wrong to Right (Reading Content Area) 125
Figure 18. Pre and Post Z by Mean Wrong to Right at the School Level (Reading Content Area) 126
Figure 19. Post Erasure Z by Pre Erasure Z at the School Level (Reading Content Area) 127
Figure 20. Relationship of Covariate for HGLM by Content Area 128

About this Dissertation

Rights statement

• Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.

School	Laney Graduate School
Department	Educational Studies
Degree	PhD
Submission	Dissertation
Language	English
Research Field	Education, Tests and Measurements
Palavra-chave	tests high stakes tests high stakes assessment education validity person fit
Committee Chair / Thesis Advisor	Engelhard, George, Emory University
Committee Members	Jensen, Robert J, Emory University Cheong, Yuk Fai, Emory University

Última modificação

Primary PDF

Thumbnail	Title	Date Uploaded	Actions
	Differential Person Functioning ()	2018-08-28 11:53:44 -0400	Press to Select an action Download

Supplemental Files

Thumbnail	Title	Date Uploaded	Actions