Reinforcement Learning for Algorithmic Trading: Double Deep-Q Learning and Reinforced Deep Kalman Filters

Oxford-Man Institute Quantitative Finance Seminar Series

Oxford-Man institute welcomes Prof. Sebastian Jaimungal - University of Toronto to give a talk as part of the OMI Quantitative Finance seminar series.



Much research into optimal execution uses stringent model assumptions and applies continuous time stochastic control to solve them. Here, we instead take a model free approach and develop a variation of Deep Q-Learning to estimate the optimal actions of a trader. The model is a fully connected Neural Network trained using Experience Replay and Double DQN with input features given by the current state of the limit order book, other trading signals, and available execution actions, while the output is the Q-value function estimating the future rewards under an arbitrary action. We apply our model to nine different stocks and find that it outperforms the standard benchmark approach on most stocks using the measures of (i) mean and median out-performance, (ii) probability of out-performance, and (iii) gain-loss ratios.


Kalman filters have found use in many applications across engineering, finance, economics, and a host of other fields. The linear Gaussian assumption required to make them work, however, is rather restrictive. Krishnan (2015) introduce a variation where the observed and latent states evolution are non-linear transformations using a deep neural net called Deep Kalman Filters (DKFs). Here, we develop a reinforcement learning architecture combining probabilistic inference for learning control with DKF to solve stochastic control problems in environments without assumptions on model dynamics. The method is data efficient, robust to noisy and incomplete data, and results in superior performance in comparison to a number of other state-of-the-art methods applied to an optimal execution problem.


Prof. Jaimungal is the Director of the professional Masters of Financial Insurance program in the Department of Statistical Sciences. He is the Chair for the SIAM activity group in Financial Mathematics and Engineering, a managing editor of Quantitative Finance, and an associate editor at SIAM Journal on Financial Mathematics, among others. His current research interest include algorithmic trading, machine learning, mean-field games, stochastic control,  and commodities markets.


Sebastian Jaimungal (University of Toronto)

Wednesday, March 6, 2019 - 16:00
to 17:00

Register Online