Adaptive Approaches to Utility Computing for Scientific Applications Pubblico

Abstract

Coupling scientific applications to heterogeneous computational targets requires specialized expertise and enormous manual effort. To simplify the deployment process, we propose a novel adaptive approach that helps execute unmodified applications on raw computational resources. Our method is based on situation-specific "adapter" middleware that builds up target capabilities to fulfill application requirements, avoiding homogenization that may conceal platform-specific features. We investigate three dimensions of adaptation: performance, execution paradigm, and software deployment and propose the ADAPT framework as a methodology and a toolkit that automates execution-related tasks. For parallel applications, ADAPT matches logical communication patterns to physical interconnect topology and improves execution performance by reducing use of long-distance connections. In a proof-of-concept demonstration of application-platform paradigm transformation, ADAPT enables execution of unmodified MPI applications on the Map-Reduce Platform as a Service cloud by recreating and emulating missing MPI capabilities. To facilitate software deployment, ADAPT automatically provisions resources by applying soft-install adapters that dynamically transform target capabilities to satisfy application requirements. As a result of these types of transformations, a broader spectrum of resources can smoothly execute scientific applications, which brings the notion of utility computing closer to reality.

Table of Contents

1 Introduction 1

1.1 Goals 2
1.1.1 Executability Enhancement 2
1.1.2 Performance Adaptation 2
1.1.3 Deployment Adaptation 2
1.1.4 Paradigm Adaptation 3
1.2 Contributions 3
1.2.1 Executability Enhancement 4
1.2.2 Performance Adaptation 4
1.2.3 Deployment Adaptation 5
1.2.4 Paradigm Adaptation 6
1.3 Outline 6
2 Research Overview 7
2.1 Research Issues 10
2.2 Adapters 13
2.3 Applying Adapters to Programs 15
2.4 Related Work 16
3 Background and Context 19
3.1 An Illustrative Example 20
3.2 CFD Simulations on Different Platforms 20
3.3 Test Applications 21
3.3.1 First Test Case: Reaction-Diffusion Equation 22
3.3.2 Second Test Case: Incompressible Navier-Stokes Equations 22
3.3.3 The Organization of the Program 24
3.4 Deployment Experiences 26
3.4.1 Summary of the Packages Used in LifeV 26
3.4.2 Four Heterogeneous Target Platforms 27
3.4.3 Porting Experiences 29
3.5 Performance Evaluation 33
3.5.1 RD Test 33
3.5.2 Placement Group Benchmark for RD 34
3.5.3 Navier-Stokes Test 35
3.5.4 Cost Analysis 36
3.6 Experiences and Additional Research 36
4 Parallel MPI Applications Performance Adaptation 43
4.1 Background 43
4.2 Mapping Parallel Components into Processing Elements 44
4.3 Problem Description 47
4.3.1 Test Case: Blood Flow in a Cerebral Aneurysm 47
4.3.2 Offline Mesh Partitioning 49
4.3.3 Modeling the Communication Patterns in the Application 52
4.4 Evaluation 53
4.4.1 Message Passing Patterns 54
4.4.2 Correlation of Data Exchange with Partitioning 55
4.4.3 Evaluation Procedure 55
4.5 Results and Discussion 58
4.6 Conclusions 61
4.7 Contribution 62
5 Cost Adaptation for Time Critical Application Execution 65
5.1 Background 65
5.2 Related Work 66
5.3 Cross-Platform Cost and Utility Comparison 67
5.4 Metrics 68
5.5 Experimental Results 69
5.5.1 Performance, Scaling, and Time to Completion 70
5.6 Summary 77
6 Adaptive Deployment Automation 79
6.1 Background 79
6.2 Related Work and Concepts 81
6.3 Automatic Deployment Description 83
6.3.1 Execution Model 86
6.3.2 Deployment Model 87
6.3.3 Repository and Object Oriented Mapping 88
6.4 Example 90
6.5 Conclusion 91
6.6 Contribution 91
7 Application Paradigm Adaptation 99
7.1 Background 99
7.2 Related Work 101
7.3 MPI-MRE Execution Environment 103
7.3.1 Idea 103
7.3.2 MapReduce MPI Library 105
7.3.3 MPI-MRE Runtime 106
7.3.4 The MPI Environment 107
7.4 Results 108
7.5 Discussion 113
7.6 Contribution 114
8 Summary 117
Bibliography 121
Books and Journals 121
Electronic Resources 132

About this Dissertation

Rights statement

• Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.

School	Laney Graduate School
Department	Math and Computer Science
Degree	PhD
Submission	Dissertation
Language	English
Research Field	Computer Science
Parola chiave	Utility computing High-performance computing Cloud computing Hemodynamic simulations Application deployment Scientific and Engineering applications
Committee Chair / Thesis Advisor	Sunderam, Vaidy S, Emory University
Committee Members	Mandelberg, Kenneth I, Emory University Veneziani, Alessandro, Emory University

Ultima modifica

Primary PDF

Thumbnail	Title	Date Uploaded	Actions
	Adaptive Approaches to Utility Computing for Scientific Applications ()	2018-08-28 10:56:14 -0400	Press to Select an action Download

Supplemental Files

Thumbnail	Title	Date Uploaded	Actions