Enriching and Evaluating Meaning Representations Public

Abstract

Meaning representations receive increasing attention in the field of computational linguistics in recent years. Works include developing frameworks to represent the meaning of a sentence and exploring schemes to extract document-level interpretations. Abstract Meaning Representation (AMR) is a semantic graph framework that fails to adequately represent a number of important semantic features, including number (singular and plural), definiteness, quantifiers, and intensional contexts. Several proposals have been made to improve the representational adequacy of AMR by enriching its graph structure. However, these modifications are rarely implemented on existing AMR corpora due to the labor costs associated with manual annotation. In addition to sentence-level, there are attempts to extend such representations to the document-level, one of which is on coreference resolution. In this paper, I develop an automated annotation tool that algorithmically enriches AMR graphs to better represent number, (in)definite articles, quantificational determiners, and intensional arguments. I compare the automatically produced annotations to gold-standard manual annotations and show that the automatic annotator achieves impressive results, even matching those of human annotators for certain tasks. Through implementing the enriched structure to the large AMR 3.0 corpus and training models using the enriched graphs, I attested the feasibility of my proposals for enrichment. Additionally, I develop an annotation scheme for document-level coreference and conduct a comparison study for the text type effects across news, fables, and a novel Reddit data. The experiment results indicate the need to develop schemes adjusted for each text type due to their distinct characteristics in language use and content. 

Table of Contents

1 Introduction 1 

   1.1 Introduction.................................... 1 

   1.2 Thesis Statement ................................. 4 

2 Related Works 6 

    2.1 Related Work................................... 6 

       2.1.1 Previous AMR Graph Enrichments ................... 7 

       2.1.2 Automatic Enrichment Efforts for AMR ................ 10 

    2.2 Document-level Coreference ........................... 11 

3 Sentence-level Graph Structure 13 

    3.1 Enriched Graph Structure ............................ 13 

       3.1.1 Number .................................. 14 

       3.1.2 Articles .................................. 15 

       3.1.3 Quantifiers................................. 16 

       3.1.4 Intensionality ............................... 17 

    3.2 The Automatic Annotator ............................ 18 

       3.2.1 Number .................................. 18 

       3.2.2 Articles .................................. 19 

       3.2.3 Quantifiers................................. 19 

       3.2.4 Intensionality ............................... 19 

    3.3 Mapping Difficulties ............................... 20 

       3.3.1 Relational and Agentive/Patient Nouns. . . . . . . . . . . . . . . . . 20 

       3.3.2 Name, Date, and Quantity Entities ................... 21 

       3.3.3 Intensional Transitive Verbs ....................... 22 

       3.3.4 Other Intensional Operators....................... 22 

    3.4 Annotation Experiments ............................. 23 

       3.4.1 Method .................................. 23 

       3.4.2 Manual Annotation Results ....................... 24 

       3.4.3 Automatic Annotation Results...................... 24 

       3.4.4 Analysis of Errors............................. 26 

    3.5 Parsing Experiment................................ 27 

       3.5.1 Method .................................. 27 

       3.5.2 Results................................... 28 

       3.5.3 Analysis of Errors............................. 29 

    3.6 Discussion..................................... 31 

4 Document-level Coreference 33 

    4.1 Annotation Schemes ............................... 33 

    4.2 Methods...................................... 35 

    4.3 Results and Analysis ............................... 36 

       4.3.1 Inter-Annotator Agreement ....................... 36 

       4.3.2 Challenges................................. 37 

       4.3.3 Analysis.................................. 38 

5 Conclusion 

About this Honors Thesis

Rights statement

• Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.

School	Emory College
Department	Quantitative Science
Degree	B.S.
Submission	Honors Thesis
Language	English
Research Field	Language, Linguistics Computer Science
Mot-clé	Meaning Representation Computational Linguistics Automatic Data Enrichment Annotation
Committee Chair / Thesis Advisor	Choi, Jinho D. , Emory University
Committee Members	Xiong, Li, Emory University Pak, Marjorie , Emory University

Dernière modification

Primary PDF

Thumbnail	Title	Date Uploaded	Actions
	Enriching and Evaluating Meaning Representations ()	2022-04-09 21:48:46 -0400	Press to Select an action Download

Supplemental Files

Thumbnail	Title	Date Uploaded	Actions