Using Rasch Measurement Theory to Validate the Student Performance Character and Student Moral Character Scales Open Access
Caines, Jade A. (2011)
Published
Abstract
Abstract
According to Bond & Fox (2001), without deliberate, thoughtful,
and scientific
construction of measures, psychosocial research will progress
slowly. In the field of character
education, Davidson , Khmelkov, and Lickona (2010) contend that assessment should be
a high
priority for practitioners. Heeding their advice, this study
further investigates the validity and
gender invariance of two existing scales that measure two types of
character within students:
S tudent Performance Character (SPC) and Student Moral
Character (Cornerstone
Consulting &
Evaluation, 2009). Using data
collected on 239 middle-school students through a character
education evaluation, this study addresses three research
questions:
1. How well do items on the Student Performance Character and
Student Moral Character
scales measure the two constructs?
2. Do the items on the Student Performance Character and Student
Moral Character scales
fit the Rasch measurement model framework? If so, how well is the
fit?
3. Do any of the items display gender differential item functioning
(DIF)? If so, what are
their patterns, directions, and magnitudes?
Results indicate mixed results for both scales regarding rating
scale diagnostics, model-
data fit and item fit to constructs. Overall, the SPC scale shows a
better fit to the Rasch model
framework than the SMC scale; principal component analyses suggest
multidimensionality for
the SMC scale. Finally, no gender DIF is evident. Implications of
the results for scale revisions
and theoretical advancement are discussed.
Table of Contents
TABLE OF CONTENTS
CHAPTER I
.................................................................................................................................................................1
STATEMENT OF THE
PROBLEM.........................................................................................................................4
PURPOSE
....................................................................................................................................................................5
RATIONALE...............................................................................................................................................................6
CHAPTER II
.............................................................................................................................................................16
THEORETICAL
FRAMEWORK...........................................................................................................................16
RASCH MEASUREMENT THEORY
................................................................................................................16
VALIDITY THEORY
.....................................................................................................................................17
CHAPTER
III............................................................................................................................................................20
REVIEW OF
LITERATURE...................................................................................................................................20
DEFINING
CHARACTER...............................................................................................................................20
DEFINING CHARACTER EDUCATION
...........................................................................................................23
SCALES/INSTRUMENTS BASED PRIMARILY ON MORAL DEVELOPMENT THEORY
(MORAL
CHARACTER-RELATED)...................................................................................................................................................25
Moral Judgment Scale.
.......................................................................................................................25
Defining Issues
Test............................................................................................................................27
Moral Judgment Test (MJT).
..............................................................................................................31
Prosocial Moral Reasoning Scale
(PROM)........................................................................................31
Ethics Position Questionnaire.
...........................................................................................................32
SCALES/INSTRUMENTS BASED PRIMARILY ON PSYCHOLOGICAL THEORIES
(PERFORMANCE
CHARACTER-RELATED)...................................................................................................................................................32
Responsibility: Personal Responsibility Measure
(PRM)...................................................................33
Responsibility: Perceived Responsibility for Learning Scale.
............................................................33
Motivation: Student Motivation Scale (SMS).
....................................................................................34
Motivation: Motivated Strategies for Learning Questionnaire (MSLQ).
...........................................34
Motivation: Academic Motivation
Scale.............................................................................................34
Motivation: School Achievement Motivation Rating
Scale.................................................................35
Persistence: Persistence Scale for
Children.......................................................................................35
Persistence: The Urgency, lack of Premeditation, lack of
Perseverance, Sensation seeking impulsive behaviour scale (UPPS).
....................................................................................................................36
Persistence: The Eysenck I.6 Junior Impulsiveness Subscale.
...........................................................37
Self-control: The Brief Self-Control Scale (BSCS).
............................................................................37
Self-control: The Self-Control Schedule
(SCS)...................................................................................38
Self-efficacy: The Self-Efficacy Scale.
................................................................................................39
Diligence: The Diligence
Inventory....................................................................................................40
SCALES/INSTRUMENTS BASED ON MORAL AND PERFORMANCE CHARACTER
THEORY..............................40
Individual and Team Character in Sport Questionnaire (ITSQ).
.......................................................40
Student Performance Character and Student Moral Character
Scales..............................................41
SUMMARY
..................................................................................................................................................44
CHAPTER IV
............................................................................................................................................................48
METHODOLOGY....................................................................................................................................................48
INSTRUMENT
..............................................................................................................................................48
PARTICIPANTS AND SETTING
......................................................................................................................51
PROCEDURES..............................................................................................................................................52
Gender and grade differences
............................................................................................................53
Rating scale diagnostics
.....................................................................................................................53
Model data fit
.....................................................................................................................................55
Reliability
statistics.............................................................................................................................56
Chi-square statistics
...........................................................................................................................56
Mean square error
statistics...............................................................................................................57
Dimensionality analyses
.....................................................................................................................58
Differential item functioning (DIF)
analyses......................................................................................58
LIMITATIONS
..............................................................................................................................................58
CHAPTER
V..............................................................................................................................................................60
RESULTS...................................................................................................................................................................60
GENDER AND GRADE DIFFERENCES
...........................................................................................................60
RATING SCALE DIAGNOSTICS FOR SPC
SCALE...........................................................................................60
Orientation of items (SPC)
.................................................................................................................61
Frequency of use (SPC).
.....................................................................................................................61
Distribution of observations (SPC).
...................................................................................................61
Monotonicity of average measures (SPC).
.........................................................................................62
Category fit (SPC).
.............................................................................................................................62
Monotonicity of thresholds
(SPC).......................................................................................................62
Distance from the thresholds (SPC).
..................................................................................................63
Log-likelihood chi-square (SPC).
.......................................................................................................63
RATING SCALE DIAGNOSTICS FOR SMC SCALE
.........................................................................................63
Orientation of items (SMC)
................................................................................................................63
Frequency of use (SMC).
....................................................................................................................64
Distribution of observations
(SMC)....................................................................................................64
Monotonicity of average measures (SMC).
........................................................................................64
Category fit (SMC).
............................................................................................................................64
Monotonicity of thresholds
(SMC)......................................................................................................65
Distance from the thresholds (SMC).
.................................................................................................65
Log-likelihood chi-square (SMC).
......................................................................................................66
MODEL DATA FIT
.......................................................................................................................................66
Reliability
statistics.............................................................................................................................66
Mean square error
statistics...............................................................................................................67
Variable
maps.....................................................................................................................................67
SUMMARY
..................................................................................................................................................68
DIMENSIONALITY ANALYSES
.....................................................................................................................68
Investigation of explained
variance....................................................................................................68
Examination of residual
plots.............................................................................................................68
DIFFERENTIAL ITEM FUNCTIONING (DIF)
ANALYSIS..................................................................................69
DISCUSSION.............................................................................................................................................................71
RESEARCH QUESTION 1
..............................................................................................................................71
RESEARCH QUESTION 2
..............................................................................................................................75
Student Performance Character item
fit.............................................................................................75
Student Moral Character item
fit........................................................................................................75
Dimensionality....................................................................................................................................76
RESEARCH QUESTION 3
..............................................................................................................................77
IMPLICATIONS
............................................................................................................................................78
RECOMMENDATIONS
..................................................................................................................................79
SUMMARY
..................................................................................................................................................80
TABLE
1.....................................................................................................................................................................82
GILLIGAN'S THEORY OF MORAL DEVELOPMENT
.......................................................................................82
TABLE
2.....................................................................................................................................................................83
KOHLBERG'S THEORY OF MORAL DEVELOPMENT
.....................................................................................83
TABLE
3.....................................................................................................................................................................84
EXAMPLE OF KOHLBERG'S MORAL JUDGMENT MANUAL
ITEM..................................................................84
TABLE
4.....................................................................................................................................................................85
PSYCHOLOGICAL REALMS OF MORAL AND PERFORMANCE CHARACTER
...................................................85
TABLE
5.....................................................................................................................................................................86
STUDENT PERFORMANCE CHARACTER AND STUDENT MORAL CHARACTER ITEMS
...................................86
TABLE
6.....................................................................................................................................................................88
STUDENT PERFORMANCE CHARACTER ITEMS BY PSYCHOLOGICAL
REALM...............................................88
TABLE
7.....................................................................................................................................................................90
STUDENT MORAL CHARACTER ITEMS BY PSYCHOLOGICAL
REALM...........................................................90
TABLE
8.....................................................................................................................................................................92
SUMMARY STATISTICS OF STUDY SAMPLE
.................................................................................................92
TABLE
9.....................................................................................................................................................................94
MEANS FOR GENDER GROUPS ON SPC AND
SMC......................................................................................94
TABLE
10...................................................................................................................................................................95
MEANS FOR MORAL CHARACTER AND GRADES
.........................................................................................95
TABLE
11...................................................................................................................................................................96
MEANS FOR PERFORMANCE CHARACTER AND GRADES
.............................................................................96
TABLE
12...................................................................................................................................................................97
STUDENT PERFORMANCE CHARACTER ITEM FIT STATISTICS
.....................................................................97
TABLE
13...................................................................................................................................................................98
STUDENT PERFORMANCE CHARACTER CATEGORY STATISTICS
.................................................................98
TABLE
14...................................................................................................................................................................99
STUDENT MORAL CHARACTER ITEM FIT STATISTICS
.................................................................................99
TABLE
15.................................................................................................................................................................100
STUDENT MORAL CHARACTER CATEGORY STATISTICS
...........................................................................100
TABLE
16.................................................................................................................................................................101
PERSON RELIABILITIES
.............................................................................................................................101
TABLE
17.................................................................................................................................................................102
ITEM
RELIABILITIES..................................................................................................................................102
TABLE
18.................................................................................................................................................................103
STUDENT PERFORMANCE CHARACTER ITEM FIT VALUES (RSM)
............................................................103
TABLE
19.................................................................................................................................................................104
STUDENT MORAL CHARACTER ITEM FIT VALUES (RSM)
........................................................................104
TABLE
20.................................................................................................................................................................105
STUDENT PERFORMANCE CHARACTER ITEM FIT VALUES (PCM)
............................................................105
TABLE
21.................................................................................................................................................................106
STUDENT MORAL CHARACTER ITEM FIT VALUES (PCM)
........................................................................106
TABLE
22.................................................................................................................................................................107
STUDENT PERFORMANCE CHARACTER RESIDUAL VARIANCE (IN EIGENVALUE
UNITS)............................107
TABLE
23.................................................................................................................................................................108
STUDENT MORAL CHARACTER RESIDUAL VARIANCE (IN EIGENVALUE UNITS)
.......................................108
TABLE
24.................................................................................................................................................................109
STUDENT PERFORMANCE CHARACTER DIF
ANALYSES............................................................................109
TABLE
25.................................................................................................................................................................110
STUDENT MORAL CHARACTER DIF
ANALYSES........................................................................................110
FIGURE
1.................................................................................................................................................................111
STUDENT PERFORMANCE CHARACTER WRIGHT MAP
(RSM)...................................................................111
FIGURE
2.................................................................................................................................................................112
STUDENT PERFORMANCE CHARACTER WRIGHT MAP
(PCM)...................................................................112
FIGURE
3.................................................................................................................................................................113
STUDENT MORAL CHARACTER WRIGHT MAP (RSM)
..............................................................................113
FIGURE
4.................................................................................................................................................................114
STUDENT MORAL CHARACTER WRIGHT MAP (PCM)
..............................................................................114
FIGURE
5.................................................................................................................................................................115
STUDENT PERFORMANCE CHARACTER RESIDUAL VARIANCE SCREE
PLOT..............................................115
FIGURE
6.................................................................................................................................................................117
STUDENT MORAL CHARACTER RESIDUAL VARIANCE SCREE
PLOT..........................................................117
FIGURE
7.................................................................................................................................................................119
STUDENT PERFORMANCE CHARACTER PLOT OF STANDARDIZED RESIDUAL PCA
(CONTRAST1) .............119
FIGURE
8.................................................................................................................................................................121
STUDENT MORAL CHARACTER PLOT OF STANDARDIZED RESIDUAL PCA (CONTRAST
1)........................121
FIGURE
9.................................................................................................................................................................123
STUDENT MORAL CHARACTER PLOT OF STANDARDIZED RESIDUAL PCA (CONTRAST
2)........................123
REFERENCES
........................................................................................................................................................125
About this Dissertation
School | |
---|---|
Department | |
Degree | |
Submission | |
Language |
|
Research Field | |
Keyword | |
Committee Chair / Thesis Advisor | |
Committee Members |
Primary PDF
Thumbnail | Title | Date Uploaded | Actions |
---|---|---|---|
|
Using Rasch Measurement Theory to Validate the Student Performance Character and Student Moral Character Scales () | 2018-08-28 16:06:26 -0400 |
|
Supplemental Files
Thumbnail | Title | Date Uploaded | Actions |
---|