Automatic Generation of Emotion-based Responses to User Profiles Público
Zhang, Xinman (Spring 2021)
Published
Abstract
While existing conversational agents are able to generate grammar-correct responses, the responses are often lack of personal expressions when users start to talk about themselves. This thesis aims to improve the engagement and naturalness of responses when responding to user profiles, such as personality traits, preferences, and experiences. Our main idea is to embed the emotion factors into the response generation task given user profiles. We first developed a profile classifier to identify whether a sentence is a profile or not. We also developed an emotion classifier with 32 emotion labels, which allows us to detect the emotion of any sentence. Then, we posted profiles extracted by the profile classifier and collected corresponding responses in two parts including exclamation and follow up. Using this dataset, we presented two models that concatenated the response emotion predicted by the emotion classifier either before or after the given utterance. The result shows that concatenating the desired emotion before the given utterance generates better results for the exclamation prediction task, and concatenating the desired emotion after the given utterance generates better results for the follow up prediction task. Overall, the generated responses are encouraging. Since the profile classifier allows us to extract profiles from a large dataset, it will be easy to generate more input data to the model and thus improve the quality of responses in the future.
Table of Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2 Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2.1 Seq2Seq Model With RNN . . . . . . . . . . . . . . . . 5
2.2.2 Transformer . . . . . . . . . . . . . . . . . . . . . . . . 6
3 Profile Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.1.1 Persona Chat . . . . . . . . . . . . . . . . . . . . . . . 10
3.1.2 Reddit Comments . . . . . . . . . . . . . . . . . . . . . 10
3.1.3 My Customized Dataset . . . . . . . . . . . . . . . . . 12
3.2 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.3 Experiments and Evaluation . . . . . . . . . . . . . . . . . . . 13
3.4 Results and Analysis . . . . . . . . . . . . . . . . . . . . . . . 14
3.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4 Emotion Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.2 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.2.1 Train on situation . . . . . . . . . . . . . . . . . . . . . 20
4.2.2 Test with utterance . . . . . . . . . . . . . . . . . . . . 20
4.3 Experiments and Evaluation . . . . . . . . . . . . . . . . . . . 21
4.4 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . 22
4.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
5 Emotion-based Response Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
5.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
5.1.1 Data Collection From Mturk . . . . . . . . . . . . . . . 28
5.1.2 Apply Emotion Classifier . . . . . . . . . . . . . . . . . 31
5.2 Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
5.2.1 Categorize Personas . . . . . . . . . . . . . . . . . . . 32
5.2.2 Categorize Follow Ups . . . . . . . . . . . . . . . . . . 33
5.2.3 Similarity Between Responses . . . . . . . . . . . . . . 36
5.3 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
5.3.1 Embed Emotions . . . . . . . . . . . . . . . . . . . . . 37
5.3.2 Evaluation Methods . . . . . . . . . . . . . . . . . . . 38
5.4 Experiments and Evaluation . . . . . . . . . . . . . . . . . . . 39
5.5 Results and Analysis . . . . . . . . . . . . . . . . . . . . . . . 42
5.5.1 Model-Level Analysis . . . . . . . . . . . . . . . . . . . 43
5.5.2 Task-Level Analysis . . . . . . . . . . . . . . . . . . . . 44
5.5.3 Approach-Level Analysis . . . . . . . . . . . . . . . . . 44
5.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
6 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
6.1 MTurk More Profiles . . . . . . . . . . . . . . . . . . . . . . . 46
6.2 Desired-Emotion Predictor . . . . . . . . . . . . . . . . . . . . 47
7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
About this Honors Thesis
School | |
---|---|
Department | |
Degree | |
Submission | |
Language |
|
Research Field | |
Palavra-chave | |
Committee Chair / Thesis Advisor | |
Committee Members |
Primary PDF
Thumbnail | Title | Date Uploaded | Actions |
---|---|---|---|
|
Automatic Generation of Emotion-based Responses to User Profiles () | 2021-04-13 23:42:18 -0400 |
|
Supplemental Files
Thumbnail | Title | Date Uploaded | Actions |
---|