Reinforcement Learning with Manifold Optimization Pubblico
Hasnain, Syed Jafer (Spring 2024)
Published
Abstract
Reinforcement learning is a rapidly advancing field which in recent years has expanded out of its niche origins to become one of the most potent paradigms available within machine learning. The explosion of deep learning in the past decade has only furthered this growth, leading an acceleration in the development of state-of-the-art RL methods for highly complex use cases. Simultaneously, the study of information geometry has expanded the understanding of the geometric spaces traversed by RL algorithms as the field matures. The first part of this work provides an introduction to reinforcement learning from its first principles, followed by an exploration of two categories of differential/information geometric methods applied to RL. The experiments in this work aim to broaden the understanding of this manifold structure and draw intuitive conclusions regarding the utility of such techniques for improving RL methods.
Table of Contents
1 Introduction
2 Background
2.1 Introduction to Reinforcement Learning
2.1.1 History and examples
2.1.2 Markov decision processes
2.2 Methods and algorithms
2.2.1 Classifications
2.2.2 Dynamic programming
2.2.3 Temporal-difference methods
2.2.4 Function approximation
2.2.5 Policy-gradient methods
3 Manifold-optimized RL
3.1 Motivation and previous work
3.2 Manifold-restricted methods
3.2.1 Orthogonal Procrustes
3.2.2 Curvilinear descent on Stiefel manifold
3.2.3 Experiments and Results
3.3 Manifold-regularized/preconditioned methods
3.3.1 Motivation
3.3.2 Geometry of statistical manifolds
3.3.3 Natural policy-gradient
3.3.4 Empirical Fisher information matrix
3.3.5 Approximate-Fisher natural policy-gradient
3.3.6 Experiments and Results
A Appendix
Bibliography
About this Honors Thesis
School | |
---|---|
Department | |
Degree | |
Submission | |
Language |
|
Research Field | |
Parola chiave | |
Committee Chair / Thesis Advisor | |
Committee Members |
Primary PDF
Thumbnail | Title | Date Uploaded | Actions |
---|---|---|---|
|
Reinforcement Learning with Manifold Optimization () | 2024-04-22 17:27:25 -0400 |
|
Supplemental Files
Thumbnail | Title | Date Uploaded | Actions |
---|