Much work on AI for finance considers single agent problems. Typically, the agent is a trader who applies a policy (trading strategy) to play a repeated game against “the market” which is typically viewed as an abstract, static environment adhering to identifiable, limited memory dynamics. Although such abstractions can sometimes lead to successful trading strategies in certain regimes, they blend out the intrinsic multi-agent nature of financial markets, populated by a multitude of intelligent agents, with varying degrees of rationality, that might learn to adapt their strategies to serve their own benefit.
To model this perspective more accurately can be beneficial both from a perspective of single agent trading strategy design and from a global multi-agent perspective that aims to study and engineer financial market dynamics.
Firstly, in the long term, an agent embedded in this multi-agent market environment may find it insufficient to learn a static policy. Instead, it may wish to employ algorithms capable of learning to compete well in anticipation of adversarial, adaptive behavior. This can be of value in particular in scenarios where trades have price impact, resulting from opposing market participants learning to anticipate objectives and strategies and to exploit them to their own advantage.
Secondly, studying intelligent trading algorithms and their ensuing market dynamics from a multi-agent perspective can lead to many worthwhile research directions, with possible findings of relevance to policy makers and practitioners alike. Interesting questions to investigate from game theoretic and mechanism design point of views are manifold and include examples such as studying the impact of different market rules to the dynamics of intelligent agents, the effect of tax interventions as well as changes and robustness of price discovery in the presence of failures as well as of intelligent artificial agents employing strategies of increasing levels of sophistication.
Considering increasing prevalence of AI in finance and electronic trading, we believe multi-agent research to be of profound importance to understand modern financial markets. Current research is nascent, but we have recently decided to grow our profile in this area with several academics and students researching the field. Active projects include the following:
I. Market simulator (MAXE) for agent-based modelling (ABM)
In agent-based modelling (ABM), one studies the effect of different types of agent behavior, market rules and anomalies on market dynamics by virtue of large scale simulation of populations of agents at the order book level. We are developing MAXE [LINK], an efficient market exchange simulator that can serve as a testbed for simulation of large multi-agent systems at scale .
II. Agent-based modelling to study reinforcement learning in limit order books
Modern reinforcement learning algorithms often give rise to highly complex nonlinear policies. We will study their robustness and the properties of price dynamics of limit order books inhabited by orders set by such learning algorithms. Of particular interest is in how far the learning capabilities can result in stabilising or destabilising the dynamics and how reinforcement learning algorithms might be designed or adapted to promote desirable properties of global market dynamics. Other projects will investigate the capabilities of neural policies to discover, exploit or extend well-known high-frequency strategies when agents learn to trade against each other in order books.
- Peter Belcak, Jan-Peter Calliess and Stefan Zohren, Fast Agent-Based Simulation Framework with Applications to Reinforcement Learning and the Study of Trading Latency Effects, Accepted at Workshop for Multi-Agent-Based Simulation (MABS), AAMAS, 2021.