Automatic Generation of Emotion-based Responses to User Profiles Open Access

Abstract

While existing conversational agents are able to generate grammar-correct responses, the responses are often lack of personal expressions when users start to talk about themselves. This thesis aims to improve the engagement and naturalness of responses when responding to user profiles, such as personality traits, preferences, and experiences. Our main idea is to embed the emotion factors into the response generation task given user profiles. We first developed a profile classifier to identify whether a sentence is a profile or not. We also developed an emotion classifier with 32 emotion labels, which allows us to detect the emotion of any sentence. Then, we posted profiles extracted by the profile classifier and collected corresponding responses in two parts including exclamation and follow up. Using this dataset, we presented two models that concatenated the response emotion predicted by the emotion classifier either before or after the given utterance. The result shows that concatenating the desired emotion before the given utterance generates better results for the exclamation prediction task, and concatenating the desired emotion after the given utterance generates better results for the follow up prediction task. Overall, the generated responses are encouraging. Since the profile classifier allows us to extract profiles from a large dataset, it will be easy to generate more input data to the model and thus improve the quality of responses in the future. 

Table of Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.2 Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.2.1 Seq2Seq Model With RNN . . . . . . . . . . . . . . . . 5

2.2.2 Transformer . . . . . . . . . . . . . . . . . . . . . . . . 6

3 Profile Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.1.1 Persona Chat . . . . . . . . . . . . . . . . . . . . . . . 10

3.1.2 Reddit Comments . . . . . . . . . . . . . . . . . . . . . 10

3.1.3 My Customized Dataset . . . . . . . . . . . . . . . . . 12

3.2 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.3 Experiments and Evaluation . . . . . . . . . . . . . . . . . . . 13

3.4 Results and Analysis . . . . . . . . . . . . . . . . . . . . . . . 14

3.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

4 Emotion Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.2 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4.2.1 Train on situation . . . . . . . . . . . . . . . . . . . . . 20

4.2.2 Test with utterance . . . . . . . . . . . . . . . . . . . . 20

4.3 Experiments and Evaluation . . . . . . . . . . . . . . . . . . . 21

4.4 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . 22

4.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

5 Emotion-based Response Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

5.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

5.1.1 Data Collection From Mturk . . . . . . . . . . . . . . . 28

5.1.2 Apply Emotion Classifier . . . . . . . . . . . . . . . . . 31

5.2 Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

5.2.1 Categorize Personas . . . . . . . . . . . . . . . . . . . 32

5.2.2 Categorize Follow Ups . . . . . . . . . . . . . . . . . . 33

5.2.3 Similarity Between Responses . . . . . . . . . . . . . . 36

5.3 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

5.3.1 Embed Emotions . . . . . . . . . . . . . . . . . . . . . 37

5.3.2 Evaluation Methods . . . . . . . . . . . . . . . . . . . 38

5.4 Experiments and Evaluation . . . . . . . . . . . . . . . . . . . 39

5.5 Results and Analysis . . . . . . . . . . . . . . . . . . . . . . . 42

5.5.1 Model-Level Analysis . . . . . . . . . . . . . . . . . . . 43

5.5.2 Task-Level Analysis . . . . . . . . . . . . . . . . . . . . 44

5.5.3 Approach-Level Analysis . . . . . . . . . . . . . . . . . 44

5.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

6 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

6.1 MTurk More Profiles . . . . . . . . . . . . . . . . . . . . . . . 46

6.2 Desired-Emotion Predictor . . . . . . . . . . . . . . . . . . . . 47

7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

About this Honors Thesis

Rights statement

• Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.

School	Emory College
Department	Computer Science
Degree	B.S.
Submission	Honors Thesis
Language	English
Research Field	Computer Science
Keyword	Natural Language Processing Response Generation
Committee Chair / Thesis Advisor	Jinho Choi, Emory University
Committee Members	Manuela Manetta, Emory University Davide Fossati, Emory University

Last modified

Primary PDF

Thumbnail	Title	Date Uploaded	Actions
	Automatic Generation of Emotion-based Responses to User Profiles ()	2021-04-13 23:42:18 -0400	Press to Select an action Download

Supplemental Files

Thumbnail	Title	Date Uploaded	Actions