Chapter 3: Understanding the Trading Exchange Dynamics

In the previous chapter, we learned how to create High-Frequency Trading (HFT) systems. We focused intensely on the critical components of a trading system. We also reviewed in detail how to create an order book, which is basically a replication of what an exchange collects from all the trading participants. In this chapter, we are going to study how an exchange works.

We will describe the functional components of an exchange and we will focus in depth on the matching engine. Understanding how the matching engine of an exchange works is one of the most important tasks you will have to do when creating HFT strategies.

This chapter will cover the following topics:

• Understanding trading exchanges
• Understanding matching engines
• Architecting a trading exchange

In Chapter 2, The Critical Components of a Trading System, we gained a decent idea of how to design a trading system. We went over how to design a book, create trading signals, and receive a market response in great detail. In this chapter, we will explain in depth how an exchange works.

Architecting a trading exchange for handling orders at a large scale

Existing owners can deal with potential purchasers on stock exchanges. Exchanges are not primary markets: they can be secondary, tertiary markets. Companies that trade on stock exchanges don't buy and sell their own assets every day. They may buy back stock or issue new stocks when necessary. In a stock exchange, we purchase stocks from another shareholder. When we sell stocks, we sell them to another investor.

History of trading exchanges

In the 16th and 17th centuries, the first stock exchanges arose in Europe, mostly in port towns or commerce centers such as Antwerp, Amsterdam, and London. However, because a small number of corporations did not issue equity, these early stock markets were more analogous to bond exchanges. Most early corporations were deemed semi-public enterprises since governments had to allow them to conduct business.

The New York Stock Exchange (NYSE), enabling equity trading, first appeared in America in the late 18th century. The Philadelphia Stock Exchange (PHLX) is credited as America's first stock exchange. With the signing of the New York Stock Exchange Act in 1792, the NYSE was born.

With the introduction of contemporary stock markets, a new era of regulation and professionalization began, ensuring that buyers and sellers of stocks could trust that their transactions would be completed at acceptable prices and within a reasonable time frame. Today, there are several stock markets in the United States and across the world, many of which are electronically linked. As a result, markets have become more efficient and liquid. Stocks are, of course, the most well-known traded asset classes; however, foreign exchange, fixed income, future, options, crypto, and many other types of asset classes are also traded.

A stock exchange's share prices can be determined by a variety of methods. Conducting an auction, in which buyers and sellers make bids and offers to purchase or sell, is the most common approach. An offer (or ask) is the price at which someone wishes to purchase something, whereas a bid is the price at which they wish to sell something. When the bid and ask are equal, a transaction is made.

Understanding features of an exchange

A stock exchange (exchange) has a lot of diverse features. An exchange is a marketplace that brings together various market players to streamline transactions, decrease risks, and help in price discovery. There are several components in an exchange. The following are the primary systems, in general:

• Listings: These are the corporations that are traded by market participants on the exchange. They are essentially private enterprises that go through the initial public offering procedure to become public. Valuation, liquidity, and compliance expenses are all important factors to consider when choosing an exchange to list on.
• Matching engine: This would be similar to the old pit, where brokers stood about yelling instructions to each other. It's now fully automated as a matching engine algorithm, which takes care of the transactions. In the market, the engine publishes the order book (pending trades) and matches them properly. The speed at which these deals are matched and completed varies and is measured in nanoseconds. The way the trading engine determines the prices varies slightly between exchanges. In the General order book and matching engine section, we will explain it more in depth.
• Post-trade: Payment and settlement, as well as trade reconciliation, are all part of this process to guarantee that all orders are correctly matched and completed. Essentially, this is the tedious (but necessary) backend job.
• Market data: The exchange handles large amounts of data. It is sold to a variety of market participants. Trade prices, trading volumes, firm announcements/filings, and so on are all examples of this. Co-location (as we defined in Chapter 1, Fundamentals of a High-Frequency Trading System) has also become widespread because of HFT. As a result, the speed at which this data may be accessed is also marketed.
• Market participants: Clearing and trading members are the market participants. Each has its own set of qualifications, with clearing members having stricter restrictions.

Members also place collateral with the exchange to protect themselves in the event of a member's collapse. Because deals involve two parties, the exchange will step in to perform the trade with the second party if one side defaults. As a result, collateral is necessary for the exchange to assert a claim against the defaulting party while also maintaining market stability. Brokers and proprietary trading houses are the most common trading members. Clearing members are major participants who assist in the clearing of deals.

• Regulation: Depending on the asset class, the regulations differ. Different exchanges will have different regulations, depending on the jurisdiction. This is done to prevent money laundering and market tampering such as insider trading and market manipulation.

Exchanges can also monitor company announcements to ensure that all required disclosures are made, promoting a transparent marketplace. In addition to managing market participants, the exchange must also deal with internal compliance and government authorities.

We have learned what features an exchange must have; now we will discuss in detail the architecture of an exchange.

Exchange architecture

Trading platforms are responsible for executing orders received from buy-side portfolio managers, managing and monitoring orders during the execution process, and offering electronic access to a number of venues. On the sell side, support is required for processing customer orders and maintaining trading positions.

An exchange architecture provides buy/sell trading capabilities and must meet the following business requirements:

• Support front, middle, and back office trading capabilities as well as basic and complicated rule-based and algorithmic trade techniques.
• Support backtesting and live execution of the preceding strategies throughout the development lifecycle.
• Display trading and blotter UIs (desktop applications, web-based/mobile apps).
• Support a trading as a service (TaaS) business model that might be delivered as a utility using open APIs. We talked about the integration with the trading system and the exchange using API working with the FIX protocol in the previous chapter.
• Support global integration with a wide range of external parties.
• Support a wide range of financial products.
• Should be highly scalable.

We are presenting the main functionality of an exchange in the following figure (we represented three queues for three companies: Tesla, Microsoft, and Apple):

Figure 3.1 – Trading exchange architecture

In this figure, we can see a Trading System T1 connected to the exchange. As we explained in Chapter 2, The Critical Components of a Trading System, the trading system initiates two connections with the exchange: price updates and orders. When an order is sent to the exchange, it will follow these steps:

1. Depending on the asset class and the instrument, it will be routed to queues. Each queue is created for one given price and one given symbol.
2. The matching engine handles the orders one at a time.
3. If there is a change in the order book (handled by the matching engine), the change will be communicated to the trader and also given to all the market participants (every update will be sent to the clearing/post trades if a trade occurs).

It is easy to observe that the communication between traders and exchanges must be fast-paced to be able to reach the speed for HFT. Therefore, the choice of protocol to convey a message is critical. A string-based protocol such as the FIX protocol is not sufficient. Most of the exchanges where it is possible to trade within microseconds use binary protocols. Trading systems are designed to provide data very quickly to trading strategies. For an exchange, the goal is to provide data to the matching engine. We are going to describe in depth what the matching engine algorithm is in the next section.

General order book and matching engine

Millions of investors and traders make up the total market, all of whom may have various opinions on the worth of a particular stock and, as a result, the price at which they are willing to purchase or sell it. Over the course of a trading day, the thousands of transactions that occur when these investors and traders transform their intentions into actions by buying and/or selling a stock generate minute-by-minute gyrations in it.

A stock exchange provides a platform for this type of trading by connecting buyers and sellers of equities. A stockbroker is required for the typical person to have access to these markets. This stockbroker serves as a go-between for the buyer and the seller.

Initially, matching buyers and sellers of stocks on an exchange was done manually, but computerized trading systems are now being used more frequently. The open outcry system, in which dealers utilized verbal and hand-signal communication to purchase and sell large blocks of stocks in the trading pit or on the exchange floor, was the manual form of trading. It has been replaced by electronic trading platforms. These technologies can match buyers and sellers significantly more efficiently and quickly than people, leading to major advantages, including cheaper trading costs and speedier trade execution.

The intention of creating a transaction between buyers and sellers is kept in what we call an order book. This order book is the same as the one we previously described for a trading system. It contains bids and offers from all the market participants. The process, which is going to match buyers and sellers, is handled by the matching engine. This algorithm matches buy and sell orders to execute securities deals. Matching engines have different algorithms to describe how orders are matched and filled in what sequence, which varies depending on where the trade is routed.

The matching engine algorithm is depicted in Figure 3.2. The inputs are the Order (1) coming from the trader and the Order book (2) (which contains the orders already placed on the exchange). This algorithm will return the list of the Trades (3) and the list of Resting orders (4). Every order coming to the system will be processed one by one.

Figure 3.2 – Matching engine algorithm with inputs and outputs

When using HFT strategies, nanoseconds are important to be profitable. In this book, we will learn in detail how to optimize a trading system to get to the best performance possible. Meanwhile, an understanding of the exchange is required. As we previously described, an exchange is a server accepting connections from trading systems and running the matching engine algorithm on the order book being the structure collecting all the orders. Since all exchanges have their own matching algorithm, it is important to know the basic scenarios you will encounter when trading.

In all the following scenarios, we will explain what occurs in the exchange for an Order getting into the exchange in relation to the Order book. We will first learn about the most basic case, which is matching for the best price.

Best price scenario

By default, a matching engine will always try to find the best price available (2) for a given order (1).

The matching engine algorithm finds the best price available as it is written in Figure 3.3. In this figure, we can see order #1 getting into the exchange. This order will match with order #3, since the price of the order is better for the buyer. Indeed, the buyer wants to buy an asset for $100. The exchange has this asset for $99 and for $100. The matching engine will match with the available best price, which is $99:

Figure 3.3 – Best price scenario

The result of the algorithm in this context will be to have a trade at $99 between orders #1 and #3. Order #2 will remain.

In this example, the quantity was 100. We need to learn what will happen when the quantities of two matching orders are different.

Partial fill scenario

In the previous example, we had two orders matching with the same quantity. In the example in Figure 3.4, we have order #1 with a quantity of 4 and order #3 with a quantity of 1. In this situation, to fill order #1, we will need to have three more shares. This trading exchange doesn't have in its order book the amount to satisfy this transaction. Therefore, orders #3 and #2 will be filled and the remaining quantity of 1 coming from order #1 will remain on the exchange. That's why the output of the algorithm for this case is two filled orders and one order left.

Figure 3.4 – Partial fill scenario

In the two previous examples, we had matching liquidities. Indeed, for the price that was asked, we had a liquidity to match this price. We now need to study what would happen when the liquidity cannot be matched with another one.

No match scenario

In the scenario represented in Figure 3.5, we have order #1 getting into the system and the order book has two orders, #2 and #3. Since the price to buy is $98, which is way lower than the price where the participants are ready to sell, $99, the matching engine will not match any orders. Order #1 will stay on the exchange:

Figure 3.5 – No match scenario

In the previous scenario, we have different price levels in the book. We need to study what would happen if the price of the liquidities is the same.

Multiple orders with the same price

In the scenario represented in Figure 3.6, we have two orders with the same price and the incoming order with the same price. The way that the orders will be filled depends on the configuration of the matching engine:

Figure 3.6 – Multiple orders with the same price in the order book

The matching engine's algorithm is crucial in determining what kind of behavior we want to encourage in the exchange. The two most popular implementations of these algorithms will be discussed in the sections that follow.

Let's talk about the different types of algorithms.

FIFO

Time/price priority, also known as First In First Out (FIFO), is the most widely used algorithm. We know, from the exchange architecture represented in Figure 3.1, that orders are stored in a queue for a given price level. Once the orders get into the matching engine, they will be stamped with the time that they entered the system. Therefore, no orders will have the same timestamp. With this observation, when we apply the FIFO algorithm, we will match the incoming order with the one that has the lower timestamp. In this situation, order #3 has been on the exchange for a period longer than order #2. Consequently, order #3 will be matched first with the incoming order #1. Any modifications to the orders will result in their position being lost in the order of execution. Depending on the exchanging, changing the quantity of a given order will cause this order to lose its priority. But for all the exchanges, if we have a price change, the order will lose priority because it will need to change the FIFO queue.

Figure 3.7 – FIFO algorithm

Even if the FIFO algorithm is the one that is used the most, many other algorithms can be used. The last algorithm we will discuss is the pure pro-rata algorithm.

Pure pro-rata

Orders are filled using a pro-rata algorithm that considers pricing, order lot size, and time. A market participant's entering order is shared evenly among matching counterorders proportionally to their quantity.

Figure 3.8 – Pure pro-rata

Figure 3.8 shows that a buy order with a price of $100 will be executed with two orders with the same price, regardless of their timestamps. The exchange will use this algorithm to encourage participants to place orders, even if these participants are slower than the fastest participants.

Other forms of pro-rata matching

To encourage trading, the pro-rata algorithm is frequently used with other algorithms. It is generally used to incentivize particular behaviors among market participants.

Pro-rata with top-order is a method linked with pro-rata. In this situation, the oldest counter order is completed in full first, followed by a pro-rata distribution of the other counter orders, as indicated in the following diagram:

Figure 3.9 – Pro-rata algorithm variant

Figure 3.9 explains the pro-rata algorithm variant. In Figure 3.8, all the orders in the book were filled with the same quantity. In this variant, we give more weight when filling the orders to the older ones in the book. With this method, the exchange will still encourage participants to trade even if they are not fast enough, but the faster participants will be rewarded more by getting more traded quantities.

Any exotic configuration can be added to this algorithm. For instance, if we want to encourage bigger orders, a weight could be introduced by filling the orders with a larger quantity.

Let's end our coverage of the different scenarios that a matching engine can encounter. We will now wrap up this part by summarizing what we discussed.

Summary

As previously described, one nanosecond can create an edge in HFT. Knowing the microstructure of the market by learning how an exchange works (in terms of priority queues and the matching engine) will help you understand how to design your trading strategy. You now know that amending the price of an order will result in the loss of priority of the amended order in the queue. We also learned that depending on the exchange, changing the quantity of an order can get the same result. This chapter showed you how to design a trading exchange. We developed an in-depth understanding of how a matching engine works. In the next chapter, we will explain how hardware and operating systems operate in the context of HFT systems and trading exchanges.