Regularization of ill-posed inverse problems under mixed precision arithmetics Open Access

Abstract

In numerical computations, using low precision floating point arithmetic enables computer programs for scientific applications to have faster loops, less communication, and lower energy consumption. As low precision arithmetic leads to limited accuracy for certain data, modern computer architectures built on graphics processing units can be implemented using mixed precision for scientific computations. The basic idea is to use low precision arithmetic to accelerate speed on certain calculations, mixed with a limited amount of high precision calculations, while maintaining sufficiently appropriate accuracy of the final result. 

Recent work studies the use of mixed precision arithmetic for algorithms to solve certain basic, well-conditioned linear systems. In this thesis, we will extend these ideas to the more complicated class of inverse problems, where it is necessary to employ a technique known as regularization to compute an approximate solution of a severely ill-conditioned problem.

We will explore different regularization methods, such as truncated singular value decomposition, Tikhonov regularization, and methods of choosing their respective regularization parameters, under mixed precision arithmetic. We will also analyze the performance of iterative methods with different preconditioners to regularize under mixed precision arithmetic.

Table of Contents

1 Introduction 1

1.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Background 3

2.1 Floating Point Arithmetics . . . . . . . . . . . . . . . . . . . . . . . . 3

2.2 Inverse Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.3 Regularization Methods . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.3.1 Truncated SVD . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.3.2 Tikhonov Regularization . . . . . . . . . . . . . . . . . . . . . 6

2.4 Iterative Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.4.1 Iterative Refinement . . . . . . . . . . . . . . . . . . . . . . . 8

2.4.2 Iterated Tikhonov Regularization . . . . . . . . . . . . . . . . 10

2.4.3 Krylov Subspace Methods . . . . . . . . . . . . . . . . . . . . 10

2.4.4 GMRES-based iterative refinement . . . . . . . . . . . . . . . 13

3 Methods 14

3.1 Tikhonov Regularization under Mixed Precision Arithmetic . . . . . . 14

3.2 Choosing Regularization Parameters . . . . . . . . . . . . . . . . . . 15

3.2.1 Generalized Cross Validation . . . . . . . . . . . . . . . . . . . 15

3.2.2 Discrepancy Principle . . . . . . . . . . . . . . . . . . . . . . . 16

3.3 Iterative Refinement with Tikhonov Regularization . . . . . . . . . . 17

3.3.1 Non-stationary parameter . . . . . . . . . . . . . . . . . . . . 18

3.4 Preconditioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4 Numerical Experiments 21

4.1 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

4.2 Implementation of Mixed Precision Arithmetic . . . . . . . . . . . . . 23

4.3 Stopping Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4.4 Tikhonov Regularization under mixed precision . . . . . . . . . . . . 24

4.4.1 Methods selecting regularization parameters . . . . . . . . . . 26

4.5 Iterative Refinement for Tikhonov . . . . . . . . . . . . . . . . . . . . 29

4.6 Preconditioned Conjugate Gradient method . . . . . . . . . . . . . . 31

4.7 Preconditioned GMRES-IR . . . . . . . . . . . . . . . . . . . . . . . 32

5 Concluding Remarks 35

Bibliography 37

About this Honors Thesis

Rights statement

• Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.

School	Emory College
Department	Mathematics
Degree	B.S.
Submission	Honors Thesis
Language	English
Research Field	Mathematics
Keyword	mixed precision arithmetic numerical analysis inverse problem
Committee Chair / Thesis Advisor	James Nagy, Emory University
Committee Members	Clifford Carrubba, Emory University Lars Ruthotto, Emory University

Last modified

Primary PDF

Thumbnail	Title	Date Uploaded	Actions
	Regularization of ill-posed inverse problems under mixed precision arithmetics ()	2022-05-07 00:49:52 -0400	Press to Select an action Download

Supplemental Files

Thumbnail	Title	Date Uploaded	Actions