Investigating Associations between Model-based Reinforcement Learning and Model-based Navigation Open Access

Abstract

Model-based and model-free reinforcement learning and boundary-based and landmark-based learning are conceptually similar in that model-based and boundary-based systems pay attention to the overall structure and environment, while model-free and landmark-based systems focus on a reward or landmark when making a decision. The brain regions that are activated by the two reinforcement learning systems are also in parallel with the two spatial learning systems. Model-based learnings involves prefrontal cortices and hippocampi, which are also activated by boundary-based learning. Model-free learning induces activity in the dorsolateral striatum, ventral striatal projections and putamen activities, while landmark-based learning induces activity in the dorsolateral striatum. In the current study, we examined the behavioral correlation between model-based/model-free reinforcement learning and boundary/landmark based spatial learning, in order to investigate whether or not there is a domain general cognitive system that supports both model-based/boundary-based learning and model-free/landmark-based learning. Model-based and model-free learning was assessed with the two-stage decision task, and boundary and landmark-based learning was assessed with the boundary-landmark task. We tested 26 participants, and no significant correlation was found between model-based decision-making characteristics and boundary-based spatial learning. There was no significant correlation between model-free decision-making characteristics and landmark-based spatial learning. However, model-free learning indicators showed negative correlation with the average error rate in the boundary-landmark task, and the model-based indicator also showed a positive correlation with the average error rate in the boundary-landmark task. This indicates that increased reliance on the model-free decision making was associated with better performance on the spatial learning task.

Table of Contents

Table of Contents

I. Introduction 1

II. Methods 7

A. Participants 7

B. Behavioral Tasks 7

a. Two-stage Task 7

b. Boundary-Landmark Task 8

c. Scene-face Attention Task 9

d. Self-report Questionnaires 9

C. Procedures 10

III. Results 11

A. Demographics 11

B. Two-stage task 11

C. Boundary-Landmark Task 14

D. Between-Task Effects 15

a. Correlation between stay probabilities after four trial conditions in the two-stage task & errors and influence in the boundary-landmark task 15

b. Correlation between model-based characteristics in the two-stage task & errors and influence in the boundary-landmark task 16

c. Correlation between model-free characteristics in the two-stage task & errors and influence in the boundary-landmark task 17

E. Scene-Face Attention Task 18

IV. Discussion 18

A. Limitations 21

B. Future directions 22

V. Conclusion 24

References Appendices

A. Two-stage task instruction 40

About this Honors Thesis

Rights statement

• Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.

School	Emory College
Department	Psychology
Degree	BA
Submission	Honors Thesis
Language	English
Research Field	Psychology, General Psychology, Cognitive
Keyword	Boundary-based spatial learning Reinforcement learning Model-based reinforcement learning Model-free reinforcement learning Landmark-based spatial learning
Committee Chair / Thesis Advisor	Treadway, Michael T, Emory University
Committee Members	Banerjee, Samiran, Emory University Wolff, Phillip, Emory University

Last modified

Primary PDF

Thumbnail	Title	Date Uploaded	Actions
	Investigating Associations between Model-based Reinforcement Learning and Model-based Navigation ()	2018-08-28 15:50:09 -0400	Press to Select an action Download

Supplemental Files

Thumbnail	Title	Date Uploaded	Actions