What Transformers Might Know About the Physical World: T5 and the Origins of Knowledge Public

Abstract

We find that knowledge of animals and objects' physical properties does not depend on direct or indirect perceptual experience. Rather, such knowledge can emerge from inferential processes driven by the statistical properties of language. Here we investigate the latent knowledge of the T5 encoder-decoder model with respect to various physical properties of animals and objects. Such networks represent model organisms for the origins of knowledge in a learning system without innate knowledge or access to perceptual information. We proposed and evaluated three hypotheses about what T5 might know about the physical world: 1) that T5 might understand physical dimensions much like humans understand these dimensions, 2) that it has no understanding of the perceptual dimensions of experience, and 3) that it understands some dimensions of experience better than others and potentially uses the better-understood dimensions to understand the less well-understood dimensions. The results from Study 1- 4 show that knowledge of the size, weight, and shape—but not color— agrees closely with that of humans. Moreover, agreement with human judges increased with the network's size, suggesting that disagreement with humans might ultimately disappear as the size of the networks is increased. However, Study 5 shows that T5's understanding of size might rely on its understanding of weight, which supports our third hypothesis regarding T5's understanding of dimensions in physical world. 

Table of Contents

1 Introduction 1

1.1 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.2 Transformers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Experiment 10

2.1 Study 1: T5’s Knowledge in Animal Size . . . . . . . . . . . . . . .10

2.1.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11

2.1.2 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13

2.2 Study 2: T5’s Knowledge in Animal Weight . . . . . . . . . . . .14

2.2.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14

2.2.2 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15

2.3 Study 3: T5’s Knowledge in Animal Shape . . . . . . . . . . . . .15

2.3.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16

2.3.2 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18

2.4 Study 4: T5’s Knowledge in Animal Color . . . . . . . . . . . . . .19

2.4.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.4.2 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20

2.5 Study 5: T5’s Knowledge in Object Weight and Size . . . .20

2.5.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.5.2 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22

3 General Discussion 24

References 27

About this Honors Thesis

Rights statement

• Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.

School	Emory College
Department	Psychology
Subfield / Discipline	Cognition and Development
Degree	B.A.
Submission	Honors Thesis
Language	English
Research Field	Language, Linguistics Computer Science Psychology, Cognitive
Mot-clé	origins of knowledge knowledge representation T5 transformer networks natural language processing
Committee Chair / Thesis Advisor	Phillip Wolff, Emory University
Committee Members	Philip Kragel, Emory University Jinho Choi, Emory University

Dernière modification

Primary PDF

Thumbnail	Title	Date Uploaded	Actions
	What Transformers Might Know About the Physical World: T5 and the Origins of Knowledge ()	2021-04-17 14:58:05 -0400	Press to Select an action Download

Supplemental Files

Thumbnail	Title	Date Uploaded	Actions