Differential Person Functioning Público
Perkins, Aminah (2013)
Published
Abstract
The accuracy and meaningfulness of test scores is a crucial issue in educational settings marked by high-stakes assessments within No Child Left Behind and Race to the Top. Differential person functioning (DPF) is presented in this study using a Rasch measurement framework as a means for assessing the accuracy and validity of scores on educational assessments. The purpose of this study is to further our current understanding of DPF as not only a threat to test score validity, but as a way to examine individual student response behaviors. Erasure analyses and multilevel modeling are used as the methods to identify and assess DPF across various contexts. The following questions are used to guide the research:
(1) What is differential person functioning?
(2) How do the methods for assessing differential person
functioning differ across contexts?
(3) To what extent does differential person functioning contribute
to our understanding of person fit across contexts?
The first question is answered through an extensive review of literature on the various components of DPF: person measurement, person response functions, person fit indices, and response behaviors. Guiding questions (2) and (3) are explored using data from a high-stakes third grade statewide assessment of mathematics and reading achievement. These questions are explored using two case studies, each replicated within two content areas (mathematics and reading) yielding a total of four contexts that are explored. The first case study investigates the relationship between wrong-to-right erasures, person fit indices, and school-level mathematics and reading achievement using the Many Facets Rasch model (MFRM) and a pre/post erasure design. The second case study uses hierarchical generalized linear modeling (HGLM) to examine student and school factors that may be associated with the aberrant responses of students that include proficiency levels, economic status, gender, and erasure behavior.
The dissertation sheds light on the importance of evaluating DPF
when considering the validity evidence for an assessment.
Additionally, MFRM and HGLM, yielded valuable information for
researchers to begin to consider systematic routine analyses of DPF
for high stakes assessments.
Table of Contents
CHAPTER ONE: INTRODUCTION 1
Theoretical Framework 3
Statement of the Problem 8
Purpose of the Study 9
Guiding Questions 10
Definitions 10
Organization of Dissertation 12
CHAPTER TWO: REVIEW OF LITERATURE 13
Reliability of Person Measurements 13
Person Response Functions 15
Person Fit Indices 18
Response Behaviors 24
CHAPTER THREE: CASE STUDY ONE 28
Purpose 29
Research Questions 30
Methods 30
Results 35
Discussion 38
CHAPTER FOUR: CASE STUDY TWO 41
Purpose 44
Research Questions 45
Methods 45
Results 50
Discussion 54
CHAPTER FIVE: DISCUSSION & SUMMARY 56
Guiding Question #1: Differential Person Functioning 57
Guiding Question #2: The Role of Context 59
Guiding Question #3: Person Fit Across Contexts 62
Limitations 63
Implications for Research, Policy, and Practice 64
References 69
Appendix A: IRB Determination Letter 129
List of Tables
Table 1. Chronological List of Key Ideas in Person Measurement
78
Table 2. Abbreviated List of Person Fit Indices 84
Table 3. Summary of Case Studies 87
Table 4. Student Demographics for Grade 3 students in Case Study
88
Table 5. Student Erasures by School (Mathematics Content Area)
89
Table 6. Student Erasures by School (Reading Content Area) 91
Table 7. Facets Summary Statistics (Mathematics Content Area)
93
Table 8. Facets Summary Statistics (Reading Content Area) 94
Table 9. Fit Statistics for Pre and Post Erasure by School
(Mathematics Content Area) 95
Table 10. Fit Statistics for Pre and Post Erasure by School
(Reading Content Area) 97
Table 11. Mapping of Research Questions and Analysis 99
Table 12. Distribution of Aberrance 100
Table 13. Explanatory variables defined 101
Table 14. Summary Statistics from Facets Analyses 102
Table 15. Means and Standard Deviations 103
Table 16. Percentage of misfitting persons with Outfit MSE or Infit
MSE above 1.20 104
Table 17. Parameter Estimates for the Outfit Two-Level Model for
Mathematics Content Area 105
Table 18. Parameter Estimates for the Infit Two-Level Model for
Mathematics Content Area 106
Table 19. Parameter Estimates for the Outfit Two-Level Model for
Reading Content Area 107
Table 20. Parameter Estimates for the Infit Two-Level Model for
Reading Content Area 108
List of Figures
Figure 1. Impact of Crossing Person Response Functions 109
Figure 2. Illustration of the Creation of Pre-Erasure Strings
110
Figure 3. Conceptual Models 111
Figure 4. Variable Map for Students and Items (Mathematics Content
Area) 112
Figure 5. Variable Map for School, Pre/Post Indicator, Item
113
Figure 6. Variable Map for Students and Items (Reading Content
Area) 114
Figure 7. Variable Map for School, Pre/Post Indicator, Items
115
Figure 8. Mean Wrong to Right by Total Erasures at the School Level
(Mathematics Content Area) 116
Figure 9. Mean Post Erasure School Mathematics Achievement by Mean
Pre Erasure School Mathematics Achievement 117
Figure 10. Pre/Post Erasure Indicator by School Mathematics
Achievement 118
Figure 11. Pre and Post Erasure Outfit Z by Mean Wrong to Right at
the School Level (Mathematics Content Area) 119
Figure 12. Pre and Post Erasure Z by Mean Wrong to Right at the
School Level (Mathematics Content Area) 120
Figure 13. Post Erasure Z by Pre Erasure Z at the School Level
(Mathematics Content Area) 121
Figure 14. Mean Wrong to Right by Total Erasures (Reading Content
Area) 122
Figure 15. Mean Post Erasure School Reading Achievement by Mean Pre
Erasure School Reading Achievement 123
Figure 16. Pre/Post Erasure Indicator by School (Reading Content
Area) 124
Figure 17. Pre and Post Erasure Outfit Z by Mean Wrong to Right
(Reading Content Area) 125
Figure 18. Pre and Post Z by Mean Wrong to Right at the School
Level (Reading Content Area) 126
Figure 19. Post Erasure Z by Pre Erasure Z at the School Level
(Reading Content Area) 127
Figure 20. Relationship of Covariate for HGLM by Content Area
128
About this Dissertation
| School | |
|---|---|
| Department | |
| Degree | |
| Submission | |
| Language |
|
| Research Field | |
| Palabra Clave | |
| Committee Chair / Thesis Advisor | |
| Committee Members |
Primary PDF
| Thumbnail | Title | Date Uploaded | Actions |
|---|---|---|---|
|
|
Differential Person Functioning () | 2018-08-28 11:53:44 -0400 |
|
Supplemental Files
| Thumbnail | Title | Date Uploaded | Actions |
|---|