Using Rasch Measurement Theory to Validate the Student Performance Character and Student Moral Character Scales Public

Caines, Jade A. (2011)

Permanent URL: https://etd.library.emory.edu/concern/etds/m613mz32q?locale=fr
Published

Abstract

Abstract


According to Bond & Fox (2001), without deliberate, thoughtful, and scientific
construction of measures, psychosocial research will progress slowly. In the field of character
education, Davidson , Khmelkov, and Lickona (2010) contend that assessment should be a high
priority for practitioners. Heeding their advice, this study further investigates the validity and
gender invariance of two existing scales that measure two types of character within students:
S tudent Performance Character (SPC) and Student Moral Character (Cornerstone Consulting &
Evaluation, 2009). Using data collected on 239 middle-school students through a character
education evaluation, this study addresses three research questions:


1. How well do items on the Student Performance Character and Student Moral Character
scales measure the two constructs?
2. Do the items on the Student Performance Character and Student Moral Character scales
fit the Rasch measurement model framework? If so, how well is the fit?
3. Do any of the items display gender differential item functioning (DIF)? If so, what are
their patterns, directions, and magnitudes?


Results indicate mixed results for both scales regarding rating scale diagnostics, model-
data fit and item fit to constructs. Overall, the SPC scale shows a better fit to the Rasch model
framework than the SMC scale; principal component analyses suggest multidimensionality for
the SMC scale. Finally, no gender DIF is evident. Implications of the results for scale revisions
and theoretical advancement are discussed.

Table of Contents

TABLE OF CONTENTS


CHAPTER I .................................................................................................................................................................1
STATEMENT OF THE PROBLEM.........................................................................................................................4
PURPOSE ....................................................................................................................................................................5
RATIONALE...............................................................................................................................................................6
CHAPTER II .............................................................................................................................................................16
THEORETICAL FRAMEWORK...........................................................................................................................16
RASCH MEASUREMENT THEORY ................................................................................................................16
VALIDITY THEORY .....................................................................................................................................17
CHAPTER III............................................................................................................................................................20
REVIEW OF LITERATURE...................................................................................................................................20
DEFINING CHARACTER...............................................................................................................................20
DEFINING CHARACTER EDUCATION ...........................................................................................................23
SCALES/INSTRUMENTS BASED PRIMARILY ON MORAL DEVELOPMENT THEORY (MORAL CHARACTER-RELATED)...................................................................................................................................................25
Moral Judgment Scale. .......................................................................................................................25
Defining Issues Test............................................................................................................................27
Moral Judgment Test (MJT). ..............................................................................................................31
Prosocial Moral Reasoning Scale (PROM)........................................................................................31
Ethics Position Questionnaire. ...........................................................................................................32
SCALES/INSTRUMENTS BASED PRIMARILY ON PSYCHOLOGICAL THEORIES (PERFORMANCE CHARACTER-RELATED)...................................................................................................................................................32
Responsibility: Personal Responsibility Measure (PRM)...................................................................33
Responsibility: Perceived Responsibility for Learning Scale. ............................................................33
Motivation: Student Motivation Scale (SMS). ....................................................................................34
Motivation: Motivated Strategies for Learning Questionnaire (MSLQ). ...........................................34
Motivation: Academic Motivation Scale.............................................................................................34
Motivation: School Achievement Motivation Rating Scale.................................................................35
Persistence: Persistence Scale for Children.......................................................................................35
Persistence: The Urgency, lack of Premeditation, lack of Perseverance, Sensation seeking impulsive behaviour scale (UPPS). ....................................................................................................................36
Persistence: The Eysenck I.6 Junior Impulsiveness Subscale. ...........................................................37
Self-control: The Brief Self-Control Scale (BSCS). ............................................................................37
Self-control: The Self-Control Schedule (SCS)...................................................................................38
Self-efficacy: The Self-Efficacy Scale. ................................................................................................39
Diligence: The Diligence Inventory....................................................................................................40
SCALES/INSTRUMENTS BASED ON MORAL AND PERFORMANCE CHARACTER THEORY..............................40
Individual and Team Character in Sport Questionnaire (ITSQ). .......................................................40
Student Performance Character and Student Moral Character Scales..............................................41
SUMMARY ..................................................................................................................................................44
CHAPTER IV ............................................................................................................................................................48
METHODOLOGY....................................................................................................................................................48
INSTRUMENT ..............................................................................................................................................48
PARTICIPANTS AND SETTING ......................................................................................................................51
PROCEDURES..............................................................................................................................................52
Gender and grade differences ............................................................................................................53
Rating scale diagnostics .....................................................................................................................53
Model data fit .....................................................................................................................................55
Reliability statistics.............................................................................................................................56
Chi-square statistics ...........................................................................................................................56
Mean square error statistics...............................................................................................................57
Dimensionality analyses .....................................................................................................................58
Differential item functioning (DIF) analyses......................................................................................58
LIMITATIONS ..............................................................................................................................................58
CHAPTER V..............................................................................................................................................................60
RESULTS...................................................................................................................................................................60
GENDER AND GRADE DIFFERENCES ...........................................................................................................60
RATING SCALE DIAGNOSTICS FOR SPC SCALE...........................................................................................60
Orientation of items (SPC) .................................................................................................................61
Frequency of use (SPC). .....................................................................................................................61
Distribution of observations (SPC). ...................................................................................................61
Monotonicity of average measures (SPC). .........................................................................................62
Category fit (SPC). .............................................................................................................................62
Monotonicity of thresholds (SPC).......................................................................................................62
Distance from the thresholds (SPC). ..................................................................................................63
Log-likelihood chi-square (SPC). .......................................................................................................63
RATING SCALE DIAGNOSTICS FOR SMC SCALE .........................................................................................63
Orientation of items (SMC) ................................................................................................................63
Frequency of use (SMC). ....................................................................................................................64
Distribution of observations (SMC)....................................................................................................64
Monotonicity of average measures (SMC). ........................................................................................64
Category fit (SMC). ............................................................................................................................64
Monotonicity of thresholds (SMC)......................................................................................................65
Distance from the thresholds (SMC). .................................................................................................65
Log-likelihood chi-square (SMC). ......................................................................................................66
MODEL DATA FIT .......................................................................................................................................66
Reliability statistics.............................................................................................................................66
Mean square error statistics...............................................................................................................67
Variable maps.....................................................................................................................................67
SUMMARY ..................................................................................................................................................68
DIMENSIONALITY ANALYSES .....................................................................................................................68
Investigation of explained variance....................................................................................................68
Examination of residual plots.............................................................................................................68
DIFFERENTIAL ITEM FUNCTIONING (DIF) ANALYSIS..................................................................................69
DISCUSSION.............................................................................................................................................................71
RESEARCH QUESTION 1 ..............................................................................................................................71
RESEARCH QUESTION 2 ..............................................................................................................................75
Student Performance Character item fit.............................................................................................75
Student Moral Character item fit........................................................................................................75
Dimensionality....................................................................................................................................76
RESEARCH QUESTION 3 ..............................................................................................................................77
IMPLICATIONS ............................................................................................................................................78
RECOMMENDATIONS ..................................................................................................................................79
SUMMARY ..................................................................................................................................................80
TABLE 1.....................................................................................................................................................................82
GILLIGAN'S THEORY OF MORAL DEVELOPMENT .......................................................................................82
TABLE 2.....................................................................................................................................................................83
KOHLBERG'S THEORY OF MORAL DEVELOPMENT .....................................................................................83
TABLE 3.....................................................................................................................................................................84
EXAMPLE OF KOHLBERG'S MORAL JUDGMENT MANUAL ITEM..................................................................84
TABLE 4.....................................................................................................................................................................85
PSYCHOLOGICAL REALMS OF MORAL AND PERFORMANCE CHARACTER ...................................................85
TABLE 5.....................................................................................................................................................................86
STUDENT PERFORMANCE CHARACTER AND STUDENT MORAL CHARACTER ITEMS ...................................86
TABLE 6.....................................................................................................................................................................88
STUDENT PERFORMANCE CHARACTER ITEMS BY PSYCHOLOGICAL REALM...............................................88
TABLE 7.....................................................................................................................................................................90
STUDENT MORAL CHARACTER ITEMS BY PSYCHOLOGICAL REALM...........................................................90
TABLE 8.....................................................................................................................................................................92
SUMMARY STATISTICS OF STUDY SAMPLE .................................................................................................92
TABLE 9.....................................................................................................................................................................94
MEANS FOR GENDER GROUPS ON SPC AND SMC......................................................................................94
TABLE 10...................................................................................................................................................................95
MEANS FOR MORAL CHARACTER AND GRADES .........................................................................................95
TABLE 11...................................................................................................................................................................96
MEANS FOR PERFORMANCE CHARACTER AND GRADES .............................................................................96
TABLE 12...................................................................................................................................................................97
STUDENT PERFORMANCE CHARACTER ITEM FIT STATISTICS .....................................................................97
TABLE 13...................................................................................................................................................................98
STUDENT PERFORMANCE CHARACTER CATEGORY STATISTICS .................................................................98
TABLE 14...................................................................................................................................................................99
STUDENT MORAL CHARACTER ITEM FIT STATISTICS .................................................................................99
TABLE 15.................................................................................................................................................................100
STUDENT MORAL CHARACTER CATEGORY STATISTICS ...........................................................................100
TABLE 16.................................................................................................................................................................101
PERSON RELIABILITIES .............................................................................................................................101
TABLE 17.................................................................................................................................................................102
ITEM RELIABILITIES..................................................................................................................................102
TABLE 18.................................................................................................................................................................103
STUDENT PERFORMANCE CHARACTER ITEM FIT VALUES (RSM) ............................................................103
TABLE 19.................................................................................................................................................................104
STUDENT MORAL CHARACTER ITEM FIT VALUES (RSM) ........................................................................104
TABLE 20.................................................................................................................................................................105
STUDENT PERFORMANCE CHARACTER ITEM FIT VALUES (PCM) ............................................................105
TABLE 21.................................................................................................................................................................106
STUDENT MORAL CHARACTER ITEM FIT VALUES (PCM) ........................................................................106
TABLE 22.................................................................................................................................................................107
STUDENT PERFORMANCE CHARACTER RESIDUAL VARIANCE (IN EIGENVALUE UNITS)............................107
TABLE 23.................................................................................................................................................................108
STUDENT MORAL CHARACTER RESIDUAL VARIANCE (IN EIGENVALUE UNITS) .......................................108
TABLE 24.................................................................................................................................................................109
STUDENT PERFORMANCE CHARACTER DIF ANALYSES............................................................................109
TABLE 25.................................................................................................................................................................110
STUDENT MORAL CHARACTER DIF ANALYSES........................................................................................110
FIGURE 1.................................................................................................................................................................111
STUDENT PERFORMANCE CHARACTER WRIGHT MAP (RSM)...................................................................111
FIGURE 2.................................................................................................................................................................112
STUDENT PERFORMANCE CHARACTER WRIGHT MAP (PCM)...................................................................112
FIGURE 3.................................................................................................................................................................113
STUDENT MORAL CHARACTER WRIGHT MAP (RSM) ..............................................................................113
FIGURE 4.................................................................................................................................................................114
STUDENT MORAL CHARACTER WRIGHT MAP (PCM) ..............................................................................114
FIGURE 5.................................................................................................................................................................115
STUDENT PERFORMANCE CHARACTER RESIDUAL VARIANCE SCREE PLOT..............................................115
FIGURE 6.................................................................................................................................................................117
STUDENT MORAL CHARACTER RESIDUAL VARIANCE SCREE PLOT..........................................................117
FIGURE 7.................................................................................................................................................................119
STUDENT PERFORMANCE CHARACTER PLOT OF STANDARDIZED RESIDUAL PCA (CONTRAST1) .............119
FIGURE 8.................................................................................................................................................................121
STUDENT MORAL CHARACTER PLOT OF STANDARDIZED RESIDUAL PCA (CONTRAST 1)........................121
FIGURE 9.................................................................................................................................................................123
STUDENT MORAL CHARACTER PLOT OF STANDARDIZED RESIDUAL PCA (CONTRAST 2)........................123
REFERENCES ........................................................................................................................................................125

About this Dissertation

Rights statement
  • Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.
School
Department
Degree
Submission
Language
  • English
Research Field
Mot-clé
Committee Chair / Thesis Advisor
Committee Members
Dernière modification

Primary PDF

Thumbnail Title Date Uploaded Actions
Using Rasch Measurement Theory to Validate the Student Performance Character and Student Moral Character Scales () 2018-08-28 16:06:26 -0400

Supplemental Files

Thumbnail Title Date Uploaded Actions