The Evaluation of Distance Metrics for Generating Adversarial Perturbations from Univariate Time Series Data Pubblico

Abstract

Recent research has shown that data can be manipulated so that when a machine learning model analyzes the data it will almost certainly classify the input incorrectly. To produce these inputs, an attacker adds a small amount of noise to the original example that forces the model to incorrectly classify the input. The goal when creating these examples is to cause the input to be incorrectly classified by the machine learning algorithm while being nearly imperceptibly different from their benign counterparts. In the context of image processing, the amount of noise added is typically calculated using a Euclidean or infinity norm regardless of the task performed by the target model such as image classification or image segmentation. However, despite the success of such distance metrics in the image processing domain, there has been little investigation into the efficacy of these measures in the domain of time series data, where these metrics are often applied by default. In this paper, I compare the effectiveness of generating adversarial examples with a variety of distance functions targeting deep learning models to determine which are the strongest approximations for human perception. I also utilize a reader study to provide statistical evidence for the superiority of one metric over another.

Table of Contents

Contents

1 Introduction 1

Adversarial Machine Learning 1

Gaps in Existing Works 2

Contributions 3

2 Background 5

Neural Networks 5

Notation and Definitions 6

Distance Metrics 7

Threat Models 10

3 Adversarial Machine Learning Attack Algorithms 13

L-BFGS 13

Fast Gradient Sign 14

The Carlini-Wagner Attack 15

4 Methods and Experiments 17

Carlini-Wagner Attack Formulation 17

Linf based Perturbations 17

L2 based Perturbations 18

Mixed Loss Functions 18

Datasets and Models 19

Experimental Setup 21

Evaluation Metrics 22

Reader Study 23

5 Results and Analysis

Non-Mixed Attacks 25

Mixed Attacks 29

Evaluation Metrics 32

Applications to Audio Data: Preliminary Studies 34

Reader Study Results and Discussion 36

6 Conclusions, Limitations, and Future Work 39

Appendix A ResNet Architecture 42

Appendix B ECGFiveDays and ECG5000 Evaluation Metrics 44

About this Honors Thesis

Rights statement

• Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.

School	Emory College
Department	Computer Science
Degree	B.S.
Submission	Honors Thesis
Language	English
Research Field	Computer Science
Parola chiave	Deep Learning Adversarial Machine Learning
Committee Chair / Thesis Advisor	Li Xiong, Emory University
Committee Members	Ymir Vigfusson, Emory University Manuela Manetta, Emory University

Ultima modifica

Primary PDF

Thumbnail	Title	Date Uploaded	Actions
	The Evaluation of Distance Metrics for Generating Adversarial Perturbations from Univariate Time Series Data ()	2023-04-06 13:01:35 -0400	Press to Select an action Download

Supplemental Files

Thumbnail	Title	Date Uploaded	Actions