The Evaluation of Distance Metrics for Generating Adversarial Perturbations from Univariate Time Series Data Open Access
Wessell, Jack (Spring 2023)
Abstract
Recent research has shown that data can be manipulated so that when a machine learning model analyzes the data it will almost certainly classify the input incorrectly. To produce these inputs, an attacker adds a small amount of noise to the original example that forces the model to incorrectly classify the input. The goal when creating these examples is to cause the input to be incorrectly classified by the machine learning algorithm while being nearly imperceptibly different from their benign counterparts. In the context of image processing, the amount of noise added is typically calculated using a Euclidean or infinity norm regardless of the task performed by the target model such as image classification or image segmentation. However, despite the success of such distance metrics in the image processing domain, there has been little investigation into the efficacy of these measures in the domain of time series data, where these metrics are often applied by default. In this paper, I compare the effectiveness of generating adversarial examples with a variety of distance functions targeting deep learning models to determine which are the strongest approximations for human perception. I also utilize a reader study to provide statistical evidence for the superiority of one metric over another.
Table of Contents
Contents
1 Introduction 1
Adversarial Machine Learning 1
Gaps in Existing Works 2
Contributions 3
2 Background 5
Neural Networks 5
Notation and Definitions 6
Distance Metrics 7
Threat Models 10
3 Adversarial Machine Learning Attack Algorithms 13
L-BFGS 13
Fast Gradient Sign 14
The Carlini-Wagner Attack 15
4 Methods and Experiments 17
Carlini-Wagner Attack Formulation 17
Linf based Perturbations 17
L2 based Perturbations 18
Mixed Loss Functions 18
Datasets and Models 19
Experimental Setup 21
Evaluation Metrics 22
Reader Study 23
5 Results and Analysis
Non-Mixed Attacks 25
Mixed Attacks 29
Evaluation Metrics 32
Applications to Audio Data: Preliminary Studies 34
Reader Study Results and Discussion 36
6 Conclusions, Limitations, and Future Work 39
Appendix A ResNet Architecture 42
Appendix B ECGFiveDays and ECG5000 Evaluation Metrics 44
About this Honors Thesis
School | |
---|---|
Department | |
Degree | |
Submission | |
Language |
|
Research Field | |
Keyword | |
Committee Chair / Thesis Advisor | |
Committee Members |
Primary PDF
Thumbnail | Title | Date Uploaded | Actions |
---|---|---|---|
The Evaluation of Distance Metrics for Generating Adversarial Perturbations from Univariate Time Series Data () | 2023-04-06 13:01:35 -0400 |
|
Supplemental Files
Thumbnail | Title | Date Uploaded | Actions |
---|