HDSI Seminar – Xiaofei Shi- Continuous-time Reinforcement Learning with Forward-Backward Stochastic Differential Equations

When Monday April 7th 1:00pm
Where: HDSI 1st Floor Multipurpose Room 123
Title: Continuous-time Reinforcement Learning with Forward-Backward Stochastic Differential Equations

Abstract:
In this talk we introduce a mathematical formulation of reinforcement learning problem with a system of forward-backward stochastic differential equations (FBSDEs). With the Deep FBSDE Solver proposed by Han, Jentzen, and E (2018), deep architecture for FBSDE systems shows great success in continuous-time stochastic control problems. In our work, we show how to further leverage the FBSDE formulation to solve traditionally intractable equilibrium problems in finance. We present a general computational framework for solving continuous-time financial market equilibria under minimal modeling assumptions while incorporating realistic financial frictions, such as trading costs, and supporting multiple interacting agents. Inspired by generative adversarial networks (GANs), our approach employs a novel generative deep reinforcement learning framework with a decoupling feedback system embedded in the adversarial training loop, which we term as the reinforcement link. This architecture stabilizes the generator by integrating the information from the discriminator. Our theoretically guided feedback mechanism enables the decoupling of the equilibrium system, overcoming challenges that hinder conventional numerical algorithms.

Bio: Professor Xiaofei Shi is an Assistant Professor in the Department of Statistical Sciences at the University of Toronto. Before joining U of T, they worked as a Term Assistant Professor at Columbia University. Professor Shi obtained their PhD in Mathematical Finance at Carnegie Mellon University, under the supervision of Prof. Johannes Muhle-Karbe. They are mainly interested in stochastic optimization and stochastic differential equations with applications to mathematical finance and have also worked on various topics in data science, including crowdsourcing, dimensionality reduction, and sparse recovery.