Taking a Closer Look at Diagnostic Accuracy Studies Using the M-CHAT Autism Screening Tool Restricted; Files Only

Abstract

The M-CHAT is a population level autism screening tool thought to be “high quality”, but longitudinal studies suggest it is weaker than earlier cross-sectional studies suggested (Guthrie et al., 2019). This study critically analyzes diagnostic accuracy metrics from reported cross- sectional studies in a recent meta-analysis: "Sensitivity and Specificity of the Modified Checklist for Autism in Toddlers (Original and Revised)” (Wieckowski, 2023). Specifically, this study applies prevalence-based adjustments to determine the likely number of children incorrectly classified as a false negative (i.e., have autism but M-CHAT missed). Further, to address two different types of potential bias (demographic and methodological), this study also conducted analyses to determine if reported accuracy is impacted by participant background variables associated with delays in early identification (e.g., % Black) and originator effects (i.e., stronger effects for studies from original M-CHAT authors). This study was guided by Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines. Fifty articles were selected from Wieckowski (2023), 10 articles were cross-sectional population level screening studies.

Four diagnostic accuracy outcomes representing different assumptions were analyzed and compared: i) non-adjusted original, ii) epidemiological adjusted, iii) adjusting for positive screen cases missed to follow up, iv) epidemiological adjusted with positive screen cases missed to follow up. The primary hypothesis was that epidemiological adjustments will decrease sensitivity estimates similar to longitudinal studies. All analysis was conducted with functions from the Meta-Analysis of Diagnostic Accuracy (MADA) R-statistical package. Reitsma models were used to calculate pooled diagnostic metrics and predictors (i.e., background and author affiliation) and data are visualized with stratified receiver operating curves (ROC) with Guthrie et al.’s reported sensitivity as a benchmark (0.40). As hypothesized, epidemiological adjustments resulted in lower mean estimated sensitivity (0.58) followed by epidemiological positive screen adjustments (0.78), non-adjusted original (0.82), and original with positive screen adjustments (0.85). Racial/ethnic demographic was not available in most studies. The use of epidemiological adjustments can provide a more accurate representation that can improve early screening outcomes. Improving diagnostic standard measurement properties and developing appropriate thresholds for screening instruments can efficiently improve early identification efforts and lead to improving child outcomes. 

Table of Contents

Chapter 1. Introduction 1

Chapter 2. Literature Review 4

2.1 What is ASD? 4

Characteristics 4

Impact on Daily life 4

Effective Interventions 5

2.2 Evaluating ASD 5

Four stages of identifying ASD 5

Importance of screening 6

2.3 Screening Methods 7

Level one and two screening tools 8

Strengths and limitations of screening tools 9

2.4 The importance of diagnostic metrics 10

Diagnostic Accuracy 10

2.5 The M-CHAT level one autism screening tool 11

M-CHAT versions 11

M-CHAT strengths and limitations 12

2.6 Wieckowski et al. (2023) Meta-analysis and Systematic Review Study 14

2.7 Aims of the current study 16

2.8 Need Statement 17

Chapter 3. Methods 18

3.1 Identification and Inclusion Criteria of Study Samples 18

3.2 Data Extraction 18

3.3 Quality Appraisal 21

3.4 Data Approaches 23

3.5 Statistical Analysis 24

Epidemiological Adjustments 25

Missing Positive Screen Case Adjustments 26

Predictors 27

Chapter 4. Results 28

4.1 Study Characteristics 28

4.2 Missing Data Accuracy Metrics 30

4.3 Meta-analysis of Diagnostic Accuracy (MADA) 30

Original- Unadjusted 30

Original- Missing positive screen case adjustments 30

National Prevalence- Unadjusted 31

National Prevalence- Missing positive screen case adjustments 31

4.4 MADA Bivariate Analysis – Reitsma models 33

Original- Unadjusted 33

Original- Missing positive screen case adjustments 33

National Prevalence- Missing positive screen case adjustments 35

4.5 Predictors – Author Affiliation and Background 38

Chapter 5. Discussion and Public Health Implications 39

5.1 Main Findings 39

5.2 Limitations 41

5.3 Conclusion 41

5.4 Public Health Implications 42

5.5 Future Directions 43

Appendices 44

Appendix A: M-CHAT-R Screening Questions 44

Appendix B: M-CHAT-R Follow-Up Screening Questions 45

Appendix C: Wieckowski (2023) QUADAS-2 Assessment 46

Appendix D: Study Characteristics 47

Appendix E: Diagnostic Accuracy Outcome SROC Outcomes 50

Appendix F: M-CHAT Adaptations 51

References 52

About this Master's Thesis

Rights statement

• Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.

School	Rollins School of Public Health
Department	Hubert Department of Global Health
Degree	M.P.H.
Submission	Master's Thesis
Language	English
Research Field	Health Sciences, Public Health
Keyword	M-CHAT Screening tools Autism Spectrum Disorder (ASD)
Committee Chair / Thesis Advisor	Brian Barger , Georgia State University Rachel Waford, Emory University

Last modified

Primary PDF

Thumbnail	Title	Date Uploaded	Actions
	File download under embargo until 20 May 2026	2024-04-22 15:02:04 -0400	File download under embargo until 20 May 2026

Supplemental Files

Thumbnail	Title	Date Uploaded	Actions