Can NLP Models Aid in Behavioral Economics Decision-Making? Open Access
Ahmed, Junaid (Spring 2021)
Abstract
Natural Language Processing (NLP) models have seen rapid improvements in the last two years. Literature has indicated that these models are capable of reasoning, and in certain cases, reason better than humans. While behavioral economics tends to focus exclusively on human subjects, this study seeks to evaluate how NLP models fare in comparison to human subjects during cognitive bias tasks. More specifically, we evaluate how RoBERTa responds to fill-in-the-blank questions based on the conjunction fallacy. We use the conjunction fallacy due to its mathematical falsifiability and ease-of-testing. The hypothesis guiding this study is that RoBERTa outperforms human subjects. From this study, we conclude that RoBERTA does not outperform human subjects in aggregate, but shows promise for individuals prone to the conjunction fallacy, suggesting that there is value in future research. Moving forward, we recommend that other NLP models undergo similar tests across a greater range of cognitive biases to more accurately assess whether there is potential for using NLP models as external aids to decision making.
Table of Contents
1 Introduction 1
2 Background 2
2.1 The Model 2
2.1.1 Machine Learning 2
2.1.2 Natural Language Processing (NLP) 3
2.1.3 RoBERTa 3
2.1.4 Evidence for Logical and Inferential Reasoning Capabilities 5
2.1.5 Testing for Logical and Inferential Reasoning Capabilities 7
2.1.6 Analysis of Tests 14
2.2 Conjunction Fallacy 17
2.2.1 Justification 17
2.2.2 Application 18
3 Methodology 19
3.1 Hypothesis 19
3.2 Experimental Design 20
3.2.1 Questions for RoBERTa 20
3.2.2 Questions for Survey Participants 21
4 Data 22
5 Discussion 30
5.1 Results 30
5.1.1 Hypothesis Testing 30
5.2 Sources of Error 32
6 Conclusion 33
6.1 Ethical Concerns 34
7 Bibliography 35
8 Appendix 40
About this Honors Thesis
School | |
---|---|
Department | |
Degree | |
Submission | |
Language |
|
Research Field | |
Keyword | |
Committee Chair / Thesis Advisor | |
Committee Members |
Primary PDF
Thumbnail | Title | Date Uploaded | Actions |
---|---|---|---|
Can NLP Models Aid in Behavioral Economics Decision-Making? () | 2021-04-09 13:35:00 -0400 |
|
Supplemental Files
Thumbnail | Title | Date Uploaded | Actions |
---|