• —What do quants eat for dinner?
• —Depends on their risk appetite.

Many investors feel that someone can see their orders and place orders immediately ahead of them to draw liquidity and capture a small profit at the investor's expense. Often, investors sense that they can observe such market behavior in real time through a brokerage app screen. Take a quiet market; see a specific bid; place a sell order, and the bid evaporates just before the order happens to execute. How can this happen? This chapter discusses the peculiarities of front‐running in the electronic trading world we live in, as well as broader implications of liquidity, order book depth, spoofing, and more.

First, the basics—front‐running is illegal. Front‐running is defined as an activity whereby an ill‐intentioned market participant observes an incoming market order. Knowing that the order is likely to move the price just due to its basic liquidity‐taking property, the observer places a similar order directly ahead of the original investor's order. As such, the observer runs to place an order ahead or in front of the investor with the expectation of taking a better price. Next, the investor's order is executed, possibly at a worse price due to reduced liquidity, courtesy of the front‐runner. Following, the price likely moves further since the investor's order also takes out liquidity from the market. The front‐runner can now liquidate his temporary position and realize a small profit.

Thus, suppose you are an investor and want to sell 1,000 shares of IBM at the best price available, with a market order. You look at the market and see that the best bid available across all markets at the time you are placing your order is $162.96. You diligently enter your 1,000 share order into your order entry/management system (OMS), and click “Submit.” As discussed in the previous chapter, your order travels on a public network that is most‐likely unencrypted to your broker, who then decides what to do with it. Your broker may choose to match your order with opposing orders your broker has accumulated up to that point, in what's called internalization. If your order is internalized, your order never actually hits the markets; instead, you receive a confirmation of order execution without touching any of the big boards. Even when your broker chooses to internalize your order, your order may still be moving the needle on the price display in which means that you did not receive the best price you observed when you placed the order. More on this later.

Alternatively, your broker may choose to route your order to an external trading venue, such as an exchange, an ECN, a dark pool, another broker, or a third‐party market maker. And here is an important detail: When your broker sends your order on to the next execution venue, your order loses your identifier. Instead of your order being identified with your account number, your name, or your corporate identity, once your order leaves your broker's realm, your order takes on your broker's identification. The span between your computer and the order‐receiving brokerage is the only environment where you are represented and identified as yourself, be it individual investor, a large hedge fund, or another legal entity. In other words, when your order hits the financial markets at large, it does so anonymously, save for your broker's identification. Your broker uses the same order identification on all the orders it sends on to other market participants for execution. Tracing your particular order from an exchange board to your account, therefore, becomes virtually impossible, unless your broker is primarily dealing with your orders and your orders alone. Figure 4.1 illustrates the point.

Front‐running your specific orders may occur in two ways: (1) when the alleged front‐runner knows who you are, and (2) when the alleged front‐runner does not know who you are. The first case, when the alleged front‐runner is well aware of your identity, can only occur within your broker's realm, since your broker is the only party who legally has access to your identity in the financial markets (we are ruling out guys spying on your identity illegally via computer hacks here).

How can your broker front‐run you? Technically, your broker is bound by the code of best execution—that is, your broker is morally obligated (and regulated) to deliver the best possible work for the client like you and the others like you who choose to use your broker's services. Of course, the broker also needs to stay in business, and the execution business can be a tough endeavor in the age of electronic trading. Some brokers, therefore, resort to prehedging, which loosely works as follows: The broker sees your order, realizes that she may be exposed if she holds inventory and you have better information than she does (in a situation known as “adverse selection”), determines that you are likely to move the market, and jumps ahead and executes a similar order ahead of yours in the markets to ensure that she ends up on the winning side, should the markets move considerably. The broker's trade erases some of the previously available liquidity, your order gets a much worse fill than you expected, and, worst of all, the market returns back to its prior level after your order is filled, since the broker disburses the purchase for their own account following your trade. Prehedging is also known as anticipatory hedging, as in preempting and hedging the risks of impending execution.

While prehedging is currently forbidden on the CME, foreign exchange is still a wild west and equities regulators allow the use of derivatives to prehedge. Thus, once your 1,000‐share order to sell IBM hits the broker, your broker may turn to IBM options and buy 500 put options on IBM before executing your order, with the explicit purpose of protecting itself against your information asymmetry—suppose for whatever reason you know that IBM is about to crater, and your broker does not. If your broker is holding any IBM inventory, it will be at a disadvantage as the IBM price is about to sink. The seemingly innocuous options purchase by the broker has wild ramifications in today's interconnected markets. Aggressive high‐frequency traders (HFTs), discussed in the next chapter, continuously scan markets for arbitrage opportunities, and will see the temporal discrepancy between the options activity and the still‐lethargic IBM stock (your order still has not hit the markets). The HFTs will take off the price you saw when you placed the order just before your order had a chance to execute. Figure 4.2 illustrates the idea.

Are any changes for the better on the horizon? To an extent, yes. For example, in June 2016, the Bank of International Settlements, the body loosely coordinating the standards in foreign exchange transacting, proposed to eliminate prehedging from foreign exchange practices. The proposals are nonbinding at this point, and require substantial industry buy‐in to take effect, yet, this is likely a step in the right direction.

Several initiatives have tried to circumvent this situation altogether. For example, a Legal Entity Identifier (LEI) program, currently administered by the Office of Financial Research (OFR), is promoting requirements for things like mandatory trade identification, with each end‐user being a part of the record. The idea behind LEI, at least in part, is to track who is front‐running whom and help regulators make better rules about what market participants may and may not do. This initiative, conceived in the wake of Lehman Brothers' collapse in late 2009 and the mess of financial records that followed, has been adopted in markets such as swaps and insurance distribution. However, real‐time adoption of such identifiers may be far away, as the industry is still struggling to shorten its settlement cycle, often from as many as three days. In fact, as this book was written, NASDAQ announced the formation of an industry group to discuss moving settlement from the three day to a two‐day cycle (T + 3 to T + 2). As blockchain technology proliferates, however, and enables true real‐time settlement, legal entity identifiers may gain traction for trade‐by‐trade reconciliation. Still, why would a trader want to be publicly recognized on every exchange? Wouldn't that make the front‐running situation even worse? Imagine the world with all the market participants, not just brokers, knowing which trader has a high betting average?

Another possible reason for front‐running: stale quotes. In other words, the best prices you are seeing on your computer screen are simply out of date. The breakdowns of exchange feeds are still too common when sending quotes to the SEC's security information processor (SIP) tape that redistributes the best quotes back to everyone. Companies like the New York Stock Exchange (NYSE), traditional bastions of human trading, have had a hard time adjusting to new‐age technological requirements, 24/7 operational mandates for their systems, and so on. Still, even the brand‐new exchange entrants like IEX may wittingly or unwittingly be distributing stale quotes, and here's how.

IEX's innovation is to delay all orders by 350 microseconds. In all fairness, this innovation is not really IEX's: foreign exchange electronic broker EBS, a subsidiary of ICAP, introduced the 250‐microsecond‐delay loop years ago. The official reason for the delay, as explained by IEX, is that the delay stops aggressive HFTs from arbitraging price discrepancies between dark pools, giving dark‐pool prices a chance to adjust to market levels prior to execution. And that works great for ICAP in dark‐pool‐like distributed foreign exchange execution and worked fine for IEX when IEX was a dark pool. Fast‐forward to the present, IEX is a lit SEC‐registered exchange subject to national best bid/offer (NBBO), and the same delay essentially produces stale quotes and explicitly allows for front‐running.

How does that work? Suppose IEX has a limit order for IBM. IEX delays all incoming market orders by 350 microseconds (µs) “to deter high‐frequency traders”—a nonstarter measure due to the NBBO dynamics, as discussed in the next chapter. Suppose another market order has already arrived to IEX ready to claim that available liquidity that you are seeing on screen. Since the market orders are delayed, you are seeing phantom liquidity, as those orders are already spoken for by the orders that have arrived before you even looked on screen—IEX simply provides artificial or “stale” quotes by forcing delays in execution. Not only that, IEX distorts the dynamics of the entire market.

Consider this scenario: There is some unexpected news and the market is moving super‐fast. IEX has a backlog of quotes, all of which have already been spoken for by incoming market orders, sitting in their respective 350‐microsecond‐delay pens. IEX quotes are thus the best in the markets, as all of the other exchanges have moved way beyond these levels. IEX is going into the SIP (NBBO collector) as the best available quote of the moment, forcing a ton of new orders to be routed their way due to the NBBO requirement. The end result? IEX obtains a huge share of the orders by law, most of which are executed at the subpar prices, IEX captures untold commissions, and investors feel ripped‐off more than ever.

IEX introduces other opportunities for front‐running as well. All of the quotes in the SIP are already at least 1 ms delayed due to the back and forth of quote transmission and another 0.5 ms or so in SIP own quote aggregation. IEX introduces another 1 ms or so delay into the SIP quotes when accounting for data transmission speeds in excess of IEX's own delay loop. So now you have the following hierarchy: real best quotes, quotes delayed by 1 ms by IEX, and quotes delayed by 2 ms by SIP. Most of the time, markets are reasonably quiet and 1 ms delay will not matter much. However, when the markets move rapidly—for example, in response to news—the following high‐frequency arbitrage opportunity presents itself. Suppose the true best bid/offer for IBM is $150.09/$150.25, and IEX is still quoting $150.45/$150.87 into the SIP. Since the SIP‐based national best bid is at $150.45, exchanges with true market values cannot execute market orders, and instead are obligated to forward them to IEX, where the NBBO currently resides. IEX, as a result, is accumulating a backlog of market sell orders with limited liquidity to support them all. Feeding $150.00 and lower‐priced bids into the IEX system, therefore, while placing limit orders to sell at $150.09 prevailing in other markets results in virtually risk‐free short‐term arbitrage opportunity, stemming simply from IEX design.

SPOOFING, FLAKY LIQUIDITY, AND HFT

Not all that appears to be front‐running is technically that. Some perceived front‐running is flaky or vaporized limit orders—the market depth that somehow is ephemeral, easily disappearing in times of even minute stress. Not surprisingly, many investors have been blaming high‐frequency traders, and specifically, high‐frequency trading market makers, for providing this phantom liquidity and pulling it at their convenience, leaving the rest of the market in disarray.

First, the basics. High‐frequency trading (HFT) refers to a category of computer programs designed to process vast arrays of market information and trade the markets, typically in an intraday framework and only occasionally holding positions overnight. Broadly speaking, all HFT can be split into two large groups: aggressive HFT and passive HFT. The key difference between the two categories is their built‐in impatience. Aggressive HFTs tend to trade on time‐sensitive information and typically prefer to use market orders that deliver immediate execution at the best available price. Aggressive HFTs are discussed in detail in the next chapter. Passive HFTs engage in market making and other, less time‐sensitive strategies. As a result, passive HFTs mostly use limit orders. Most successful aggressive HFTs require ultra‐fast connectivity and speed of execution to reach the markets ahead of their competition.

Passive HFTs tend to reduce volatility by propping up the limit order book and reducing spreads and the bid‐ask bounce of prices. Of course, traders deploying passive HFTs can cancel their limit orders, as can everyone else placing limit orders. However, they cannot run away once their orders have been selected for matching by the exchange. In other words, just by placing a limit order, a passive HFT is committing to honor that order in the period of time before the order may be canceled. No matter how soon the order cancellation may be sent, if the limit order is the best‐priced order on the market, and if a market order arrives in the time span between the placement of the limit order and its cancellation, the limit order will be executed. Stated differently, any limit order always has a positive probability of execution. Figure 4.3 summarizes actions of passive HFTs' provision of liquidity.

Limit orders form liquidity. Liquidity is a measure of how big of an order one can place in a market, where the “market” is usually considered to be the order book for one given financial instrument. The higher the liquidity in a particular market, the less price disturbance a large order will incur there. In a perfect market with infinite liquidity, an investor may process an infinitely sized market order and not move the price 1 tick. Of course, such perfectly liquid markets tend to exist only in the imagination of academics.

In reality, in order for a buy or sell market order to be fulfilled, the market order needs to be matched with one or more limit orders of the opposite direction, buy market orders being matched with sell limit orders, and vice versa. The larger the market order, the more limit orders it will consume. Conversely, as more limit orders are available for matching the arriving market order, the larger the market order can be. Thus, in technical terms, liquidity is the set of all available limit orders that can be used for immediate execution. Figure 4.4 illustrates a snapshot of a limit order book, containing “displayed” liquidity: resting buy orders (“bids”) and sell orders (“offers”), aggregated by price from lowest to the highest. Besides displayed liquidity, most exchanges offer the opportunity to send in “hidden” limit orders that, similarly to traditional dark pools, are not revealed until they are executed.

According to folklore, modern liquidity has two subsets: “natural” liquidity and “toxic” liquidity. Natural liquidity is thought to consist of dependable limit orders ready to be matched with incoming market orders or to put it another way, liquidity placed by traders who generally plan to hold the position for longer than one day. Toxic liquidity, also referred to as opportunistic liquidity, comprises the limit orders that are not dependable or stable. Just as the toxic market order flow leaves market makers at a disadvantage in a process referred to as adverse selection, toxic liquidity can be disadvantageous to non–market‐making participants such as institutional portfolio managers. Toxic limit orders are often canceled, only to be replenished by another set of identical limit orders after a brief pause. The goal of such on‐off flickering is to be intentionally harmful to the markets along the following dimensions:

• Some market participants believe that flickering quote behavior is present to deceive market participants about the depth of the order book.
• Others believe that flickering quotes are used to prompt large traders into revealing their true position execution sizes. Such information‐mining on behalf of entities deploying flickering orders is known as phishing or pinging.
• Overall, flickering or disappearing liquidity can be toxic because it can accentuate the market impact of incoming orders. Figure 4.5 shows an example of market toxicity.

Due to the often‐intense speed of flickering observed in toxic limit orders, some consider toxic liquidity to be generated by machines more so than by humans. People are constrained; we need to physically observe and click the orders. Indeed, human market makers and institutional market participants are often described as generating the most natural liquidity. As a direct consequence, the presence of toxic liquidity has prompted debates on the usefulness of high‐frequency trading in market making. The next section discusses strategies deployed by passive high‐frequency traders and their activities in the markets.

A particular concern surrounding passive HFT has been a perceived rise in fast order cancellations and the resulting toxicity of liquidity. Some recent research went as far as to suggest that in today's markets, as many as 95 percent of all limit orders are canceled, creating wasteful clogging of networks. The same research proposed that such clogging is a result of potentially malicious activity by high‐frequency traders. New research suggests that the number of canceled orders has been significantly overstated, and, therefore, the risks associated with HFTs were blown out of proportion.

The miscalculation of the proportion of limit order cancellations happened for simple and innocuous reasons. In the exchange databases, most orders are recorded with one of the following four monikers:

• A for limit order addition
• X for limit order cancellation
• E for order execution
• P for hidden order execution

Notice that there is no code for limit order revision. Instead, when a trader sends in a request to update the price of his limit order from $33.56 to $33.58, the update is recorded as two separate transactions: a cancellation of the limit order at $33.56 and an addition of the new limit order at $33.58. When one counts X orders vis‐à‐vis the number of all orders in the database, the proportion of cancellations indeed appears to exceed 90 percent. A more detailed review, of course, presents a different picture. The vast majority of X orders are not just simple order cancellations, but are immediately replaced by another A order with a more favorable price. In other words, the majority of limit orders are not simply canceled, but are instead revised. Indeed, only 12 percent of limit orders are canceled outright without immediate replacement.

Can we be sure that a given order is canceled and then replaced? How do we know that the observed A order is not a completely separate limit order arriving and then being canceled? All orders have unique order identifiers. In the case of BATS Y‐exchange, for instance, the order identifier is a unique 12‐digit alphanumeric sequence. When an order is revised, instead of being simply canceled, the 12‐digit ID on the order cancellation (message type X) and the following re‐addition of the order (message type A) share the common identifier. Studies citing unusually large order cancellation ratios make no mention of order identification counting, likely misreading the order classification statistics.

What happens with the other 88 percent of limit orders that are not simply canceled? On BATS Y‐exchange, just 0.1 percent, or one in every thousand, of all limit orders are matched with limit orders in the limit order book—a tiny number that is a function of BATS pricing, customers, and the resulting appearance of quotes matching the NBBO in the BATS limit orders. Another 1 percent of limit orders are executed immediately upon arrival, since they hit the so‐called hidden limit orders, the limit orders of the opposite side (buy vs. sell), placed at the same or better price as the newly arriving limit orders, yet not shown in the limit order book until their execution. An increasing number of exchanges offer hidden limit orders that are a feature in the spirit of dark pools, where all the limit orders are hidden. The remaining 86 percent of all limit orders are revised, not canceled outright, until the end of the trading day.

Hidden orders are not a feature of BATS alone. Many of the exchanges have been moving in the direction of dark‐pool‐like functionality, allowing iceberg orders.

Hidden or dark orders have certainly taken their fair share of criticism over the last few years, in both dark pools and lit exchanges. Dark pool operators have been sued and accused of amoral and unscrupulous behavior and generally singled out as shady characters. People trading in the dark pools were likewise thought to be tarnished and were scrutinized for phishing, pinging, and an array of other, previously unheard of, market activities.

Specifically, phishing and pinging are two techniques traders may use to induce a behavioral response from hidden market participants in a dark pool as well as in a lit exchange utilizing hidden orders. Phishing and pinging are related concepts and work as follows. Assume there is a large dark or hidden order in the order book of the dark pool or a lit exchange. To find out where the order is and to approximate its size, a phishing trader may send out a sequence of very small orders at different price levels within the limit order book, as shown in Figure 4.6. At a certain point, the phishing orders will be picked up or executed, indicating the approximate location of the dark orders. The speed of the dark order's response to phishing orders also matters: The faster the response, the closer are the phishing orders to the dark orders—a phishing order hitting upon a price point with dark liquidity present is executed instantaneously. Think of this as slow creatures moving along a dark ocean floor—whoever drops the bait directly in front of the creatures is rewarded instantaneously with a catch!

Several dark pool and exchange operators have been screening their traders for phishing behavior. Automated Trading Desk (ATD), long since acquired by Citi, was famous for creating negative financial incentives for phishing participants in its markets. The fees ATD imposed were in proportion to the benefit the phisher received, and the fees thus worked to discourage phishing behavior. Other trading venues may terminate phishing participants altogether, but each exchange has leeway to treat their phishers differently.

What should investors do? It's worth remembering that the original purpose of dark pools and dark orders on lit exchanges was to hide large order blocks. Michael Lewis suggested that all markets are doomed and investors really have no choice but to turn to the one dark pool, IEX, that he was promoting. Aside from marketing messages, sophisticated algos help investors break down their large positions into single‐digit share or dollar trades, and process those trades in lit and well‐regulated exchanges. With an algo in hand, investors simply do not need dark pools—lit regulated exchanges work just fine. The reality is that algos are a necessity for managing many execution venue‐related risks, and should be considered a must‐have in every large investor's toolkit.

Today's most sophisticated execution algos do not just break large positions into small orders according to the VWAP or TWAP. The best execution algos take into account fees paid or received from exchanges and trading venues, and various available order types. A typical exchange may offer dozens of order types to traders of all categories, including institutions and HFTs. As of September 2015, NYSE had 25 active order types, including six types of “immediate or cancel” (IOC) orders comprising variations of a market order, five types of displayed limit orders, and four types of nondisplayed or hidden limit orders (Intercontinental Exchange 2015). Out of all the order types, NYSE IOC market‐order types comprised 32.61 percent of all orders in aggregate, displayed limit orders of all stripes accounted for 41.51 percent of all orders, and nondisplayed limit orders totaled just 2.46 percent of all orders. By comparison, the following is the distribution of orders on BATS exchanges in September 2015: BATS IOC, including vanilla market orders, occurred 13.84 percent of the time, with displayed limit order variations submitted 48.91 percent of the time, and nondisplayed orders accounting for 37.26 percent of the total order count (BATS Global Markets, 2015). The differences in order prevalence by type may be a function of market structure divergences among exchanges. However, most exchange order types have at least one commonality: the structure of order transmission to and from the exchanges.

The commonalities in order transmission are not to be confused with the language of transmission, formally known as transmission protocol. Many exchanges use FIX communication protocol to transmit messages, while some exchanges, such as NASDAQ, have proprietary data transmission models that allow information exchange to be faster and more reliable than FIX. However, most protocols deploy a message structure that includes message additions, message cancellations, and message executions, with individual messages often linked by unique order identifiers to track order arrivals and existing order modifications.

For instance, Table 4.1 shows a stylized excerpt from a message log recorded for GOOG on October 8, 2015, by BATS BYX exchange. The fields included in Table 4.1 are Unique Limit Order ID, used to identify all limit order additions and subsequent executions and revisions; the time the message was sent out by the exchange; the time when the original limit order was added; the size of the original limit order or revision; and the price of the original limit order. Table 4.1 shows two order types: A for a new limit order addition and X for limit order cancellation. Additional order message types may include partial or full executions of limit orders, market orders, and hidden order executions.

In the snippet of messages shown in Table 4.1, the first two messages pertained to order ID C91KT9003TDS. The first C91KT9003TDS message was an addition of the limit order with a price of 637.33 recorded at 9:39:01.688 AM ET. The timestamp originally was reported in milliseconds since midnight, but was converted into regular time for reader convenience. The second message pertaining to the same order ID, a cancellation, arrived just over one second later. A similar pattern occurred with the next order ID, C91KT9003UU4. The message to add the 100‐share order, this time with a price of 629.23, occurred at 9:39:09.213, while the message to cancel the same order was recorded by the exchange at 9:39:10.212, just 999 milliseconds later. The last two messages displayed in Table 4.1 are cancellations of orders placed earlier in the day and not shown in the table. On October 8, 2015, GOOG had 50,274 messages that were of one of the following types: (1) limit order additions, (2) full or partial limit order cancellations, (3) regular limit order executions, and (4) hidden order executions.

Out of those messages, 24,824 (49.3 percent) were limit order additions, 24,750 (49.2 percent) were limit order cancellations, 139 (0.3 percent) were limit order executions, and 561 (1.1 percent) were records of hidden order executions. Table 4.2 summarizes size properties of each category of orders. Out of all the added limit orders, only 49 were greater than 100 shares, and the maximum order size was 400 shares. The posted limit orders exclude hidden or dark orders that are now available in most public exchanges (“lit” markets).

After a limit order is added (message type A), it can be canceled or executed in part or in full, or it can remain resting in the order book until its expiry, typically at the end of the trading day or “until cancel.” The trader who placed the order completely determines the cancellation. The execution is a combination of factors: a resting limit order is executed when it becomes the best available order and a matching market order arrives, given that the order is not canceled before the market order arrival. A limit order may be canceled all at once or in several cancellation messages, each message chipping away at the limit order's initial size. Similarly, a limit order may be executed in full if the matching market order size is greater or equal to that of the limit order. If the limit order is larger than the matching market orders, it will be partially executed.

Table 4.3 summarizes the distributional properties of time since the last record of each order appeared. For additions of limit orders as well as for executions of hidden orders, the times are identically zero. Limit order cancellations average 8.3 seconds since the last action on the order ID: at the order placement or previous partial cancellation. The time distribution is highly skewed, with the median time between the last order action and following order cancellation of just a half a second. Executions (order types E) on average occur 18 seconds since the last order action, with the executions following limit order additions just 3 seconds at the median value.

Out of 24,824 limit orders added to GOOG on October 8, 2015, 21,698 (87 percent) were canceled in full, with just one order cancellation. On average, single cancellations arrived just five seconds after the limit order was added to the limit order book. The median shelf life of a limit order with a single cancellation was even shorter: just over half a second. Table 4.4 illustrates that most of the orders were 100 shares or smaller, but greater than 1 share. As Table 4.4 shows, most of the orders were in 100‐share lots.

The limit orders not canceled in full with a single order cancellation can be subsequently executed or canceled at a later time. Figure 4.7 displays a histogram of the number of order messages for each added limit order when the order messages exceed two (typically, addition and cancellation, or addition and execution). As Figure 4.7 shows, some limit orders end up with as many as 50 limit order cancellations.

The most interesting part of the limit order dynamics could be in the intraday evolution of orders. Until 9:28 AM ET, limit orders arrive and are promptly canceled, without any limit orders visibly resting in the limit order book for longer than five minutes. Displayed limit orders alternate between buys and sells, and various price levels. Then, at 9:28:30.231 AM ET, two orders arrive, a buy at 596.57, order ID C91KT9000RU8, and a sell at 684.27, order ID C91KT9000RU9. The buy order is left untouched until 11:52:25.912 AM, at which point the buy order is modified through a simultaneous cancellation message and another added with the same order ID and size, at 590.16. At 14:59:30.895, the same order ID is in play again, this time receiving a simultaneous cancellation message and an A message with price of 596.64. At 16:00:00, the limit order is finally canceled. The sell order C91KT9000RU9 is updated with a simultaneous cancellation and an order addition at 9:49:44.619, when the price is reset to 677.88, and then 11:56:21.674, when the price is reset to 671.49, and then 14:39:58.082, when the price is changed to 677.95. This order too is finally canceled at 16:00:00.000 by the exchange, probably because it was a day limit order.

When a limit order is adjusted, it is recorded not as a separate message, but as a sequence of two messages with the same order ID: an order cancellation followed by an immediate order addition with revised characteristics. In GOOG data for October 8, 2015, 4,794 messages existed pertaining to limit order adjustments, comprising 9.5 percent of the total message traffic. An average revision occurred 30 seconds after the last order update, indicating likely human direction. Out of all the revisions, 99.0 percent occurred within 40 seconds from the original order addition or last revision. Table 4.5 summarizes the distribution of inter‐revision times for limit orders on GOOG on October 8, 2015.

Out of all order revision traffic messages, only 488 (10.2 percent) referred to singular order updates; the remaining (89.8 percent) of revised orders incurred several sequential revisions in a row. For example, the limit sell order C91KT9003EDZ was revised five times within six seconds from 9:38:05.139 to 9:38:11.424, with the limit sell price dropping with each consecutive order from 641.26 to 641.25 to 641.14 to 641.07 to 641.05. For the 3,244 messages pertaining to the sell order revisions, the price on 95.0 percent of the orders was revised downward (i.e., improved with each revision). Similarly, for the 1,550 buy order revision messages, the price was raised to be closer to the market in 96 percent of cases. In other words, the vast majority of the 9.5 percent of all limit order traffic comprising limit order revisions was beneficial: the limit order updates tightened spreads.

Unlike order revisions, 6,168 messages, or 12.3 percent of the 50,274 total order messages for GOOG recorded on October 8, 2015, were short‐lived flashes of liquidity that can be considered “flickering liquidity.” For example, a 100‐share buy limit order C91KT9000W09 is placed at 9:30:02.763 for 632.55 only to be canceled 678 milliseconds (ms) later without a simultaneous replacement. At 9:30:09.376, another buy limit order C91KT9000XZ0 arrives for a higher price of 638.01, and is held for precisely 1,000 ms, at which point it is also canceled without an immediate replacement. Two more buy orders turn on and off sequentially, first for 636.55 at 9:30:11.403 for 1,001 ms, and then for 629.09 at 9:30:14.422 for 5,352 ms, before a hidden order execution trade print arrives: 23 shares at 642.27 executed at 9:30:47.035. A similar dance of short‐lived quotes followed by hidden order executions continues throughout much of the trading day. Of the flickering orders, 1,622 message pairs (each flickering order comprises an order addition and an order cancellation) pertained to sell limit orders, and 1,462 pairs were on the buy side of the limit order book. Table 4.6 summarizes distribution of the shelf life of orders that are canceled without immediate replacement, and can, therefore, be considered flickering.

While the flickering orders identified in Table 4.6 are likely candidates for pings or phishes, the results present a drastically different picture from that of some previous studies on the dynamics of limit orders, who find that 95.0 percent of limit orders are pings canceled within one minute of their addition.

Of the remaining 78.2 percent of the entire message traffic not accounted for in order revisions and pings, only 700 orders (1.3 percent of the total daily message traffic) were order executions. Out of those, only 139 orders (0.3 percent) were executions of limit orders displayed in the limit order book–message types E. The remaining 561 executions (1.11 percent of total message traffic) were type P messages–matches of market orders with hidden limit orders, special order types that do not appear in the centralized limit order book.

The finding that most order executions are accomplished with hidden limit orders is not entirely surprising. Some studies find that hidden orders accounted for 20.4 percent of all executions. This percentage is probably increasing as lit exchanges are moving toward structures akin to dark pools.

ORDER‐BASED NEGOTIATIONS

According to some recent research, market participants may use lit limit orders to signal their willingness to buy and sell at specific prices. Most of the execution, however, happens in the interaction with hidden or dark liquidity that cannot be directly observed in the limit order book. A swift negotiation may follow an indication of interest, resulting in a hidden order execution. A hypothesis can be put forth that the institutions and other market order and hidden order traders are influenced by flickering, suboptimal, liquidity provided by high‐frequency traders. Next, we present simple tests of the order interactions in today's markets.

To test the interaction of various order types, each order message within the data set is separated and labeled into one of the following categories: a message revision, a ping, a regular limit order addition, and a regular limit order cancellation. The message revision orders are picked out by matching limit order IDs of sequential orders where the order addition follows order cancellation with slightly different parameters. Pings are identified as order cancellations following order additions with the same order ID without subsequent order additions.

We tested two frequencies of order interactions: high, sampling orders every 10 exchange messages, and low, with considering the impact of orders every 300 messages.

The observed impact of various order types appears to change considerably from high‐frequency to lower frequency. On average throughout the day, 10 exchange messages were timestamped every five seconds, with a median time of two seconds, and the lowest decile of 67 milliseconds. Conversely, 300 messages were processed every 2.5 minutes, on average, with a median processing time falling to 1.8 minutes, and 10 percent of all 300‐message blocks crowding into 1 minute. Although a human trader can theoretically follow every 10 trading messages in just 2 seconds, a more likely scenario is that speed is processed by a machine, whereas human traders would more likely observe data at a minute scale (i.e., 300‐message horizon).

At higher (10‐message) frequencies, both regular market order executions and hidden order executions exhibit dependence on the dynamics of other order types of the following nature:

1. Flickering orders bear little impact on the execution of hidden orders—at higher frequencies, the rate of execution of hidden orders remains unchanged whether flickering orders are present or not. However, flickering orders appear to deter market order traders from sending in the market orders.
2. Limit order revisions do not change the dynamics of incoming market orders, but increase the rate of hidden order execution. Potentially, limit order revisions serve to identify hidden orders and approach hidden orders faster, resulting in matching.
3. Regular limit order placement and cancellation has the greatest impact on the execution of both market orders and hidden orders. Surprisingly, in the cases of market orders and hidden orders, the impact of new limit order arrivals and cancellations is negative: the more regular (nonrevision, no‐flicker) limit orders arrive or are canceled in the limit order book, the fewer market orders and hidden orders are executed. Potentially, new limit orders are simply alternatives to market orders, with traders choosing limit orders whenever the impending market movement is not perceived as urgent. Similarly, additions and cancellations of regular limit orders may delay hidden order discovery, reducing the hidden order cancellation rates.

At the 10‐message frequency, both hidden and lit order execution is determined by factors unrelated to the order messages immediately preceding execution. At a lower 300‐message frequency, a much stronger dependency exists between the preceding pings and order revisions. Specifically, at lower frequencies, flickering orders have a stronger impact on market and hidden order execution. An increase in pings leads to an increase of market orders and hidden order executions with 99.9 percent confidence. This finding starkly contrasts with findings about the flickering order impacts at higher frequencies where the execution of market orders declines with increases in flickering quotations.

It is as if, at lower frequencies, traders are drawn to flickering orders, while at higher frequencies, traders are repelled by pings. If higher frequency orders are machine‐generated, and lower‐frequency ones are placed by humans, then human traders are disproportionally hooked on pings, while machines find ways to ignore or even run away from them.

Similarly, at lower frequencies, limit order revisions present a much stronger influence on increased market order and hidden order execution than at higher frequencies. Finally, at lower frequencies, while the impact of regular order addition on market and hidden order executions is present, it is less statistically significant than that observed at higher frequencies—human traders do not care about the depth of the order book as much as machines do, potentially exposing the human limits to absorbing limit order book information.

The divide in how market participants perceive and interpret flickering quotes is informative on many levels. First, it could reveal a weakness in the SIP tape, administered by the SEC. The routine operation of SIP involves gathering quotes from various trading venues, finding the best bid and the best offer among the quotes, and then redistributing the best quotes back to market participants. Trading venues might use SIP to determine to which exchange to forward a market order in the absence of best quotes on a given exchange. The presence of flickering quotes on a particular exchange could cause SIP to post the flickering order as the best nationwide quote, and cause a spike in market order routings to that exchange. As a result, the routed market orders may or may not be filled up at best prices. Alternatively, people watching market data on screens could perceive the flickering quotes as the true available liquidity and attempt to execute against the quotes using either market or hidden orders.

Finally, flickering orders could be pure pings seeking to identify pools of hidden liquidity within the spread in a given limit order book. In this case, a small match of a flickering order with a hidden order establishes the location of a potential liquidity pool in the limit order book.

In the context of signaling, both hypotheses postulated at the beginning of this section appear to hold true: (1) machine traders identify and filter behavior of other machines, disregarding issues such as flickering quotes or pings, and (2) lower‐frequency traders appear to interact with flickering liquidity. While the results presented in this chapter are a case study of an individual stock, GOOG, on just one trading day, October 8, 2015, the results are easily extended to a larger stock universe where similar conclusions hold.

CONCLUSIONS

Front‐running is an often misconstrued problem. Due to the anonymity of exchanges, most perceived front‐running is due to the practice of pre‐hedging often used by brokers to diversify the risk of information asymmetry. Diversifying brokers helps investors protect their trading decisions from front‐running by avoiding information leakage via the single broker.

Contemporary equity markets are evolving to best meet institutional investors' needs in other areas of market execution. Some issues, however, particularly those pertaining to the collaboration of human and machine traders, remain unresolved. Most regulated (lit) exchanges are accommodating the demand for block trading by converging to a model that supports large hidden block orders, producing substantial liquidity readily available to execute institutional investors' mandates. In BATS data, for instance, the vast majority of order executions are conducted with hidden limit orders, and just a small fraction is carried on with market orders.

HFT market making, often considered the worst due to the flickering liquidity it delivers, is only a small fraction of available liquidity. While not as copious as previously thought, flickering liquidity appears to have a dual impact at distinct frequencies. At high frequencies, the flickering liquidity is mostly detrimental to itself, as it is readily observed and avoided by other high‐frequency market participants. At lower frequencies, however, flickering liquidity appears to attract execution of both market and hidden orders, potentially causing order routing toward flickering order books by the SEC's consolidated tape via SIP and disadvantaging human traders.

Finally, regular limit order additions and, separately, cancellations appear to deter the execution of market and hidden orders. The observed negative impact of order additions and order cancellations is more statistically significant at higher frequencies. Traders observing the markets may want to be aware of the market's responses to individual orders and reconsider their processing of market data as well as placement of their orders with the market signaling context in mind.

END OF CHAPTER QUESTIONS

1. What is liquidity?
2. What is passive HFT?
3. What is prehedging?
4. What is pinging?
5. What order types are out there?