Can NLP Models Aid in Behavioral Economics Decision-Making? Öffentlichkeit

Abstract

Natural Language Processing (NLP) models have seen rapid improvements in the last two years. Literature has indicated that these models are capable of reasoning, and in certain cases, reason better than humans. While behavioral economics tends to focus exclusively on human subjects, this study seeks to evaluate how NLP models fare in comparison to human subjects during cognitive bias tasks. More specifically, we evaluate how RoBERTa responds to fill-in-the-blank questions based on the conjunction fallacy. We use the conjunction fallacy due to its mathematical falsifiability and ease-of-testing. The hypothesis guiding this study is that RoBERTa outperforms human subjects. From this study, we conclude that RoBERTA does not outperform human subjects in aggregate, but shows promise for individuals prone to the conjunction fallacy, suggesting that there is value in future research. Moving forward, we recommend that other NLP models undergo similar tests across a greater range of cognitive biases to more accurately assess whether there is potential for using NLP models as external aids to decision making.

Table of Contents

1 Introduction 1

2 Background 2

2.1 The Model 2

2.1.1 Machine Learning 2

2.1.2 Natural Language Processing (NLP) 3

2.1.3 RoBERTa 3

2.1.4 Evidence for Logical and Inferential Reasoning Capabilities 5

2.1.5 Testing for Logical and Inferential Reasoning Capabilities 7

2.1.6 Analysis of Tests 14

2.2 Conjunction Fallacy 17

2.2.1 Justification 17

2.2.2 Application 18

3 Methodology 19

3.1 Hypothesis 19

3.2 Experimental Design 20

3.2.1 Questions for RoBERTa 20

3.2.2 Questions for Survey Participants 21

4 Data 22

5 Discussion 30

5.1 Results 30

5.1.1 Hypothesis Testing 30

5.2 Sources of Error 32

6 Conclusion 33

6.1 Ethical Concerns 34

7 Bibliography 35

8 Appendix 40

About this Honors Thesis

Rights statement

• Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.

School	Emory College
Department	Economics and Mathematics
Degree	B.A.
Submission	Honors Thesis
Language	English
Research Field	Language, Linguistics Artificial Intelligence Psychology, Behavioral Sciences
Stichwort	RoBERTa judgment NLP behavior heuristic bias artificial intelligence language model computer science behavioral economics machine learning kahneman decision making cognitive science fallacy decision science linguistics psychology reasoning natural language processing behavioral science conjunction fallacy
Committee Chair / Thesis Advisor	Stephen O'Connell, Emory University
Committee Members	Mike Carr, Emory University Phillip Wolff, Emory University

Zuletzt geändert

Primary PDF

Thumbnail	Title	Date Uploaded	Actions
	Can NLP Models Aid in Behavioral Economics Decision-Making? ()	2021-04-09 13:35:00 -0400	Press to Select an action Download

Supplemental Files

Thumbnail	Title	Date Uploaded	Actions