Approximating McDowell’s Evolutionary Theory of Behavior Dynamics with Stochastic Neural Networks Open Access

Riley, Steven (Summer 2022)

Permanent URL:


Behavioral selectionism is the metaphor that learning is like evolution, where successive generations of behaviors develop to increase their demonstrated ability to obtain reinforcement. McDowell’s evolutionary theory of behavior dynamics (ETBD) is a selectionist system based on a sexually reproducing population of bitstrings that undergoes successive rounds of emission, selection, recombination, and mutation. The ETBD is consistent with quantitative behavioral findings under variable schedules of reinforcement. However, it lacks the ability to generalize across high-dimensional input spaces, and it is not biologically plausible. Two neural network implementations of the ETBD are presented, which allow for generalization and hierarchical organization of behaviors. Rather than housing a population of behaviors, these networks encode a population within their synapse weights. Network rules acting on these encoded populations are shown to approximate operations on the ETBD’s explicit populations. The networks are evaluated against twelve quantitative behavioral findings and found to diverge from the results of the ETBD. Genetic drift in the population of behaviors in the ETBD is shown to be responsible for important features of behavior records. Adding a small amount of reinforcement unconditionally at each time step is shown to approximate the effects of genetic drift and leads to convergence between net one and the ETBD’s behavior outputs. 

Table of Contents

Table of Contents

Selectionism  1

The Evolutionary Theory of Behavior Dynamics 3

Implementation of the ETBD  3

The Artificial Organism. 3

The Environment. 6

Empirical Tests of the ETBD  7

Molar behavior. 8

Bivariate Matching on Concurrent RI Schedules. 8

Molecular Behavior. 14

Quadratic Changeovers. 14

Rapid Acquisition of Responding. 14

Gaps Between the ETBD and Live Behavior 19

Problem One: Generalization of Discriminative Stimuli. 19

Problem Two: Hierarchical Behavior Organization. 21

Problem Three: Biological Plausibility. 22

ETBD vs. Other RL Algorithms 22

Goals 24

Stochastic Networks 25

Implementing the ETBD in Stochastic Networks 27

How to Map ETBD Functions 27

Outputs 27

Hidden Neurons and Update Rules 28

Phenotype Space 28

Network One 30

Properties of Network One 30

Discussion of Network One 37

Network Two 37

Properties of Network Two 37

Discussion of Network Two 41

Method 42

Subjects, Apparatus, and Materials 42

Pilot Testing: Mapping Hyperparameters Between AOs 42

Phase One. 43

Phase Two. 43

Experiment One: Bivariate Matching and Changeovers During Concurrent RI-RI Schedules 44

Experiment Two: Exclusive Preference on RR-RR Schedules 44

Experiment Three: Preference Development During Stubbs and Pliskoff (1969) Schedules 45

Results 50

Pilot Testing: Mapping Hyperparameters 50

Phase One 50

Phase Two 53

Experiment One: Bivariate Matching and Changeovers During RI-RI Schedules 53

Experiment Two: Exclusive Preference During RR-RR Schedules 57

Phase One 57

Phases Two and Three 60

Experiment Three: Preference Development During Stubbs and Pliskoff (1969) Schedules 62

Phase One 62

Phase Two 65

Results Summary 68

General Discussion 69

Genetic Drift in the ETBD  71

Modeling Genetic Drift in Net One 72

Reinforce Every Behavior 73

Evaluating the REB Hypothesis 74

Discussion of REB  79

Conclusions 81

Limitations 82

Computational and Structural Limitations 82

Evidentiary Limitations 83

Future Directions 84

Improving Net Two 84

Beyond Nets One and Two 86

Appendix A  101

Definition of Network One 101

Genetic Algorithm A  102

Relevant Differences with the ETBD  103

Definition of Network Two 103

Appendix B  105

Problem One: Bits in a Mutating Population 105

Problem Two: Tracking the Excess in the Synapse Weights 108

Discussion 109




Figure 1. Log behavior ratio of live subject (y-axis) with identifier “C7” as a function of the pattern of the last three reinforcers (x-axis) 16

Figure 2. Log behavior ratio of AOs as a function of the pattern of the last three reinforcers 16

Figure 3. A four-dimensional hypercube represents all possible four-bit genomes 30

Figure 4. Correspondence between a stochastic network and a genetic algorithm  34

Figure 5. ETBD bias as a function of the ratio between µ values of the fitness density function 50

Figure 6. Net one bias as a function of ratio between values of  on the two alternatives 51

Figure 7. Mean coefficients from fits to cGML from experiment one 53

Figure 8. Values of G from fits of changeovers to the quadric surface                                        55

Figure 9. AO Preference for the richer alternative in a pair of unequal concurrent RR-RR schedules of reinforcement 58

Figure 10. AO preference for an arbitrary alternative during equal concurrent schedules of RR reinforcement 60

Figure 11. AO preference for an arbitrary alternative during decreasing equal concurrent RR schedules 61

Figure 12. AO development of preference under Stubbs and Pliskoff (1969) schedules of reinforcement 64

Figure 13. AO preference shifts following confirmations and disconfirmations 67

Figure 14. Target class correlations with output at  after a single reinforcer is acquired at time  70

Figure 15. Outcomes from repeating experiment one and adding REB  74

Figure 16. Outputs from repeating experiment two phase one while adding REB to net one 76

Figure 17. Outputs from repeating phases two and three from experiment two while adding REB to net one 78

Figure 18. Structure of network one 101

Figure 19. Expected count of bits in the population with a given value before and after the mutation step 105



Table 1. Summary of Findings and Hypotheses from ETBD Experiments 17

Table 2. Details of Experimental Procedures 46

Table 3. Quadric Surface Coefficients and Values of Interest 55

Table 4. Summary of Results with Respect to Hypotheses 68

About this Dissertation

Rights statement
  • Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.
Subfield / Discipline
  • English
Research Field
Committee Chair / Thesis Advisor
Committee Members
Last modified

Primary PDF

Supplemental Files