Approximating McDowell’s Evolutionary Theory of Behavior Dynamics with Stochastic Neural Networks Público
Riley, Steven (Summer 2022)
Abstract
Behavioral selectionism is the metaphor that learning is like evolution, where successive generations of behaviors develop to increase their demonstrated ability to obtain reinforcement. McDowell’s evolutionary theory of behavior dynamics (ETBD) is a selectionist system based on a sexually reproducing population of bitstrings that undergoes successive rounds of emission, selection, recombination, and mutation. The ETBD is consistent with quantitative behavioral findings under variable schedules of reinforcement. However, it lacks the ability to generalize across high-dimensional input spaces, and it is not biologically plausible. Two neural network implementations of the ETBD are presented, which allow for generalization and hierarchical organization of behaviors. Rather than housing a population of behaviors, these networks encode a population within their synapse weights. Network rules acting on these encoded populations are shown to approximate operations on the ETBD’s explicit populations. The networks are evaluated against twelve quantitative behavioral findings and found to diverge from the results of the ETBD. Genetic drift in the population of behaviors in the ETBD is shown to be responsible for important features of behavior records. Adding a small amount of reinforcement unconditionally at each time step is shown to approximate the effects of genetic drift and leads to convergence between net one and the ETBD’s behavior outputs.
Table of Contents
Table of Contents
Selectionism 1
The Evolutionary Theory of Behavior Dynamics 3
Implementation of the ETBD 3
The Artificial Organism. 3
The Environment. 6
Empirical Tests of the ETBD 7
Molar behavior. 8
Bivariate Matching on Concurrent RI Schedules. 8
Molecular Behavior. 14
Quadratic Changeovers. 14
Rapid Acquisition of Responding. 14
Gaps Between the ETBD and Live Behavior 19
Problem One: Generalization of Discriminative Stimuli. 19
Problem Two: Hierarchical Behavior Organization. 21
Problem Three: Biological Plausibility. 22
ETBD vs. Other RL Algorithms 22
Goals 24
Stochastic Networks 25
Implementing the ETBD in Stochastic Networks 27
How to Map ETBD Functions 27
Outputs 27
Hidden Neurons and Update Rules 28
Phenotype Space 28
Network One 30
Properties of Network One 30
Discussion of Network One 37
Network Two 37
Properties of Network Two 37
Discussion of Network Two 41
Method 42
Subjects, Apparatus, and Materials 42
Pilot Testing: Mapping Hyperparameters Between AOs 42
Phase One. 43
Phase Two. 43
Experiment One: Bivariate Matching and Changeovers During Concurrent RI-RI Schedules 44
Experiment Two: Exclusive Preference on RR-RR Schedules 44
Experiment Three: Preference Development During Stubbs and Pliskoff (1969) Schedules 45
Results 50
Pilot Testing: Mapping Hyperparameters 50
Phase One 50
Phase Two 53
Experiment One: Bivariate Matching and Changeovers During RI-RI Schedules 53
Experiment Two: Exclusive Preference During RR-RR Schedules 57
Phase One 57
Phases Two and Three 60
Experiment Three: Preference Development During Stubbs and Pliskoff (1969) Schedules 62
Phase One 62
Phase Two 65
Results Summary 68
General Discussion 69
Genetic Drift in the ETBD 71
Modeling Genetic Drift in Net One 72
Reinforce Every Behavior 73
Evaluating the REB Hypothesis 74
Discussion of REB 79
Conclusions 81
Limitations 82
Computational and Structural Limitations 82
Evidentiary Limitations 83
Future Directions 84
Improving Net Two 84
Beyond Nets One and Two 86
Appendix A 101
Definition of Network One 101
Genetic Algorithm A 102
Relevant Differences with the ETBD 103
Definition of Network Two 103
Appendix B 105
Problem One: Bits in a Mutating Population 105
Problem Two: Tracking the Excess in the Synapse Weights 108
Discussion 109
Figures
Figure 1. Log behavior ratio of live subject (y-axis) with identifier “C7” as a function of the pattern of the last three reinforcers (x-axis) 16
Figure 2. Log behavior ratio of AOs as a function of the pattern of the last three reinforcers 16
Figure 3. A four-dimensional hypercube represents all possible four-bit genomes 30
Figure 4. Correspondence between a stochastic network and a genetic algorithm 34
Figure 5. ETBD bias as a function of the ratio between µ values of the fitness density function 50
Figure 6. Net one bias as a function of ratio between values of on the two alternatives 51
Figure 7. Mean coefficients from fits to cGML from experiment one 53
Figure 8. Values of G from fits of changeovers to the quadric surface 55
Figure 9. AO Preference for the richer alternative in a pair of unequal concurrent RR-RR schedules of reinforcement 58
Figure 10. AO preference for an arbitrary alternative during equal concurrent schedules of RR reinforcement 60
Figure 11. AO preference for an arbitrary alternative during decreasing equal concurrent RR schedules 61
Figure 12. AO development of preference under Stubbs and Pliskoff (1969) schedules of reinforcement 64
Figure 13. AO preference shifts following confirmations and disconfirmations 67
Figure 14. Target class correlations with output at after a single reinforcer is acquired at time 70
Figure 15. Outcomes from repeating experiment one and adding REB 74
Figure 16. Outputs from repeating experiment two phase one while adding REB to net one 76
Figure 17. Outputs from repeating phases two and three from experiment two while adding REB to net one 78
Figure 18. Structure of network one 101
Figure 19. Expected count of bits in the population with a given value before and after the mutation step 105
Tables
Table 1. Summary of Findings and Hypotheses from ETBD Experiments 17
Table 2. Details of Experimental Procedures 46
Table 3. Quadric Surface Coefficients and Values of Interest 55
Table 4. Summary of Results with Respect to Hypotheses 68
About this Dissertation
| School | |
|---|---|
| Department | |
| Subfield / Discipline | |
| Degree | |
| Submission | |
| Language |
|
| Research Field | |
| Palavra-chave | |
| Committee Chair / Thesis Advisor | |
| Committee Members |
Primary PDF
| Thumbnail | Title | Date Uploaded | Actions |
|---|---|---|---|
|
|
Approximating McDowell’s Evolutionary Theory of Behavior Dynamics with Stochastic Neural Networks () | 2022-07-16 17:08:04 -0400 |
|
Supplemental Files
| Thumbnail | Title | Date Uploaded | Actions |
|---|