Read with Emora: Revolutionizing Children's Reading Comprehension Through An LLM-Powered Personalized Intelligent Tutoring System for Vocabulary, Grammar, and Context-Based Learning Pubblico

Abstract

Reading comprehension is critical for educational success, yet many students in the United States struggle with low proficiency. Decades of research on reading comprehension techniques show depth-based in-context approaches for practical learning components like grammar and vocabulary, spaced repetition and practice for long-term retention, and personalized learning to ensure all students learn at the same rate. However, many educators have yet to incorporate these findings into classroom practice opting instead for core or standardized curricula, which can contribute to suboptimal student reading progression. This thesis presents the development of Read with Emora, an Intelligent Tutoring System designed to enhance current reading comprehension-focused tutoring systems by incorporating LLM-powered personalized learning. Read with Emora integrates large language models to generate AI-driven reading passages, context-based questions, and adaptive feedback tailored to each student’s grade level, vocabulary list, grammar rules, and chosen topics. By integrating LLMs for personalized instruction, Read with Emora aims to improve content generation and personalized learning features present in current reading comprehension-focused ITSs.

This project demonstrates the potential for LLM-driven tutoring systems to provide accessible and free reading comprehension tools for parents, teachers, and students. Read with Emora generates personalized reading materials and assessments that would otherwise require purchasing, making reading practice more available to students regardless of background or resources. The evaluation of Read with Emora focuses on three key areas: (1) the integration of critical components such as learning objectives, grammar rules, and vocabulary words into generated passages through assigned integration scores, (2) the accuracy of the system's grading for context-based comprehension questions compared to human evaluators (analyzed through loss between human and LLM-assigned scores), and (3) the variety of questions generated by assigning questions and analyzing their distribution across six categories. The results show that while the system excels in integrating personalized topics and generating factual recall questions, it faces challenges in more context-dependent areas such as complex grammar rules, factual content integration, and Cause and Effect questions. The system also exhibited leniency in grading, often assigning partial credit for incomplete or incorrect responses, highlighting the need for refinement in the grading prompt.

This project offers a promising innovation for improving reading comprehension systems through LLMs. Future directions include refining Read with Emora's question generation and grading systems, particularly in generating nuanced question types and improving grading accuracy, incorporating second language learning benefits, and expanding the system's feature set with tools like speech-to-text functionality and image generation. Ultimately, this thesis underscores the importance of integrating LLMs into comprehension-focused educational tools to provide children with personalized, effective learning experiences.

Table of Contents

1 Introduction 1

1.1 Reading Comprehension in Children . . . . . . . . . . . . . . . . . . . 1

1.2 LLMs in Personalized Learning Intelligent Tutoring Systems . . . . . 2

1.3 Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Background 6

2.1 Theoretical Approaches to Reading Comprehension and Personalized

Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.1.1 Current Classroom Limitations . . . . . . . . . . . . . . . . . 6

2.1.2 Learning Theories Supporting ITS Development . . . . . . . . 8

2.2 Existing Intelligent Tutoring Systems for Reading Comprehension . . 9

2.2.1 My Turn To Read . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2.2 AutoTutor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.2.3 The Reading Tutor . . . . . . . . . . . . . . . . . . . . . . . . 12

2.3 Advances in ITS with LLM and Spaced Repetition Features . . . . . 13

2.3.1 ITSs and Spaced Repetition Techniques . . . . . . . . . . . . 13

2.3.2 ITSs Utilizing LLMs for Personalized Learning . . . . . . . . . 16

2.4 Novelty of Proposed Personalized Intelligent Tutoring System: Read

with Emora . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3 Read with Emora (RwE) 21

3.1 Materials and Data Collection . . . . . . . . . . . . . . . . . . . . . . 21

3.1.1 Learning Objectives . . . . . . . . . . . . . . . . . . . . . . . 22

3.1.2 Grammar Rules . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.1.3 Vocabulary Words . . . . . . . . . . . . . . . . . . . . . . . . 31

3.2 Theoretical Approach and Core Algorithms . . . . . . . . . . . . . . . 33

3.2.1 Adaptive Scoring System . . . . . . . . . . . . . . . . . . . . . 33

3.2.2 Spaced Repetition and Learning Algorithms . . . . . . . . . . 34

3.2.3 Grammar and Vocabulary Integration . . . . . . . . . . . . . . 37

3.2.4 RwE Personalization . . . . . . . . . . . . . . . . . . . . . . . 38

3.3 Implementation of Key Algorithms and Features . . . . . . . . . . . . 41

3.3.1 Program Structure . . . . . . . . . . . . . . . . . . . . . . . . 42

3.3.2 Passage Generation . . . . . . . . . . . . . . . . . . . . . . . . 42

3.3.3 Purpose Generation . . . . . . . . . . . . . . . . . . . . . . . . 49

3.3.4 Important Sentence Identification . . . . . . . . . . . . . . . . 53

3.3.5 Question Generation . . . . . . . . . . . . . . . . . . . . . . . 57

3.3.6 Gold Standard Answer Generation . . . . . . . . . . . . . . . 60

3.3.7 Iterative Feedback and Grader AI . . . . . . . . . . . . . . . . 64

3.3.8 Scoring and Ease Factor Adjustments . . . . . . . . . . . . . . 68

3.4 Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

3.4.1 Page Layout and Navigation . . . . . . . . . . . . . . . . . . . 69

3.4.2 Data Communication and Storage . . . . . . . . . . . . . . . . 78

4 Evaluation 79

4.1 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

4.1.1 Research Questions . . . . . . . . . . . . . . . . . . . . . . . . 82

4.1.2 Research Question 1: Evaluating Passage Generation . . . . . 83

4.1.3 Research Question 2: Evaluating Question Generation and

Grading Accuracy . . . . . . . . . . . . . . . . . . . . . . . . . 85

4.1.4 Considerations for Experiment Design . . . . . . . . . . . . . 86

5 Analysis 87

5.1 Results for RQ1: Passage Generation . . . . . . . . . . . . . . . . . . 87

5.1.1 Feature Integration by Grade . . . . . . . . . . . . . . . . . . 88

5.1.2 Feature Integration by Feature . . . . . . . . . . . . . . . . . . 88

5.2 Results for RQ2: Question Generation and Grading . . . . . . . . . . 91

5.2.1 Question Distribution and Variety . . . . . . . . . . . . . . . . 92

5.2.2 Grading Discrepancies by Category . . . . . . . . . . . . . . . 92

5.2.3 Additional Findings . . . . . . . . . . . . . . . . . . . . . . . . 93

6 Conclusion 95

6.1 Evaluation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

6.1.1 Passage Evaluation . . . . . . . . . . . . . . . . . . . . . . . . 96

6.1.2 Question Evaluation . . . . . . . . . . . . . . . . . . . . . . . 97

6.2 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

6.3 Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

6.3.1 Vocabulary and Grammar . . . . . . . . . . . . . . . . . . . . 98

6.3.2 Feature Expansion . . . . . . . . . . . . . . . . . . . . . . . . 99

6.3.3 Higher Fidelity Implementations . . . . . . . . . . . . . . . . . 100

6.3.4 Second Language Learning Benefits . . . . . . . . . . . . . . . 100

6.4 Final Thoughts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

A Appendix 102

A.1 Learning Objectives by Grade . . . . . . . . . . . . . . . . . . . . . . 102

A.2 Excluded Learning Objectives by Grade . . . . . . . . . . . . . . . . 103

A.3 Grammar Rules by Grade . . . . . . . . . . . . . . . . . . . . . . . . 103

A.4 GitHub and Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

Bibliography 111

About this Master's Thesis

Rights statement

• Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.

School	Laney Graduate School
Department	Computer Science and Informatics
Degree	M.S.
Submission	Master's Thesis
Language	English
Research Field	Education, Reading Computer Science
Parola chiave	elementary education intelligent tutoring system tutoring reading comprehension large language models
Committee Chair / Thesis Advisor	Jinho Choi, Emory University
Committee Members	Li Xiong, Emory University Emily Wall, Emory University

Ultima modifica

Primary PDF

Thumbnail	Title	Date Uploaded	Actions
	Read with Emora: Revolutionizing Children's Reading Comprehension Through An LLM-Powered Personalized Intelligent Tutoring System for Vocabulary, Grammar, and Context-Based Learning ()	2024-12-04 12:36:46 -0500	Press to Select an action Download

Supplemental Files

Thumbnail	Title	Date Uploaded	Actions