Controllable Clinical Conversations: A Dialogue Act Framework for Trauma-Focused Interview Automation Öffentlichkeit
Dinh, Tung (Spring 2025)
Abstract
Access to timely and structured mental health evaluations remains a challenge, especially for individuals affected by trauma. While large language models (LLMs) offer potential for automating clinical support, most existing systems lack the structured flow and empathetic engagement required in trauma-focused diagnostic interviews. This thesis presents a novel two-stage framework for automating these interviews using dialogue act (DA) classification to improve both the coherence and clinical relevance of AI-generated responses. Our method first labels clinician utterances using a trauma-specific DA taxonomy—including categories such as Empathy (EMP), Clarification Questions (CQ), and Validation (VAL)—and then generates the next utterance based on the predicted DA tag.
We fine-tune the open-source LLaMA 3 model using Low-Rank Adaptation (LoRA) and compare its performance to GPT-4o through prompt chaining. Experiments on real-world clinical interview data demonstrate that incorporating DA tags enhances the consistency of next-utterance generation and preserves the structured flow typical of human-led assessments. Additionally, we find that limiting the model's context window improves DA classification accuracy without sacrificing response quality.
Key contributions include (1) the development of a refined DA taxonomy tailored to trauma-focused interviews, (2) a two-step generation pipeline that enables controllable and clinically aligned dialogue, and (3) evaluation on real clinical transcripts as part of the broader TraumaNLP initiative. Our findings suggest that integrating structured dialogue control into generative AI systems is a promising direction for scaling trauma assessment tools while maintaining empathy and clinical rigor.
Table of Contents
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Thesis Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.5 Key Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.6 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Background 6
2.1 Overview of Automated Mental Health Dialogue Systems . . . . . . . . . . . 6
2.1.1 Limitations of Free-Form Chatbots . . . . . . . . . . . . . . . . . . . 7
2.2 Dialogue Act Classification and Its Importance . . . . . . . . . . . . . . . . . 8
2.2.1 Gaps in Clinical Interviews . . . . . . . . . . . . . . . . . . . . . . . . 9
2.3 Automated PTSD Diagnostics and Structured Interviews . . . . . . . . . . . 10
2.3.1 Challenges in Structured Diagnostic Interviews . . . . . . . . . . . . . 10
2.4 How Our Method Addresses These Limitations . . . . . . . . . . . . . . . . . 11
3 Data 13
3.1 Dialogue Act Taxonomy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.1.1 Comparison with Existing Frameworks . . . . . . . . . . . . . . . . . 13
3.2 Annotation Guidelines and Inter-Annotator Agreement . . . . . . . . . . . . 15
i
3.3 Corpus Statistics and Qualitative Analysis . . . . . . . . . . . . . . . . . . . 16
3.4 GPT-Based Silver Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4 Models 19
4.1 System Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.2 Modeling Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.2.1 Open-Source LLaMA 3 (8B) with LoRA . . . . . . . . . . . . . . . . 20
4.2.2 GPT-4o . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.2.3 Prompt Design and Context Window . . . . . . . . . . . . . . . . . . 22
4.3 Next-Utterance Generation Approaches . . . . . . . . . . . . . . . . . . . . . 22
4.4 Dialogue Act Classification in Conversation . . . . . . . . . . . . . . . . . . . 23
4.5 Expected Outcomes and Structured Flow . . . . . . . . . . . . . . . . . . . . 23
5 Experiments 25
5.1 Dataset and Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . 25
5.1.1 Data Splits for Model Training . . . . . . . . . . . . . . . . . . . . . 25
5.1.2 Model Configuration and Experimental Environment . . . . . . . . . 26
5.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
6 Analysis 30
6.1 Quantitative Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
6.2 Qualitative Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
6.2.1 Example 1: Differentiating Empathy (EMP) vs. Simple Acknowledg-
ment (ACK) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
6.2.2 Example 2: Clarification (CQ) vs. Information-Seeking (IS) . . . . . . 31
6.2.3 Example 3: Guidance (GI) Paired with Validation (VAL) . . . . . . . 32
6.2.4 Sample Dialogue Flowchart . . . . . . . . . . . . . . . . . . . . . . . 33
6.3 Discussion of Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
6.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
7 Conclusion 35
About this Honors Thesis
| School | |
|---|---|
| Department | |
| Degree | |
| Submission | |
| Language |
|
| Research Field | |
| Stichwort | |
| Committee Chair / Thesis Advisor | |
| Committee Members |
Primary PDF
| Thumbnail | Title | Date Uploaded | Actions |
|---|---|---|---|
|
|
Controllable Clinical Conversations: A Dialogue Act Framework for Trauma-Focused Interview Automation () | 2025-04-16 23:36:41 -0400 |
|
Supplemental Files
| Thumbnail | Title | Date Uploaded | Actions |
|---|