The Science of Decision‐Making

Decision theory is the field of study focused on decision‐making. Those who work in the field tend to concentrate their efforts in one of the two main disciplines, normative and descriptive. Normative decision theory is the study of how we should make decisions. It makes heavy use of statistical analysis and requires an understanding of probabilities. Normative decisions are made objectively, without emotion and can be repeated by following a systematic pattern of behavior. The goal is a rational decision.

A decision is considered rational when the decision maker does what is most reasonable given all the relevant information available at the time the decision is made. Rational decisions are what we aspire to achieve. They are optimal. On the other hand, a decision is considered right if the outcome is at least as good as all other possible outcomes. In other words, it is considered right, because the actual outcome was successful. Importantly, rational decisions are not necessarily the same as right decisions, and the decision that winds up being right may not have been rational. The key difference between the two is that a decision can be deemed rational versus irrational at the moment it is made, whereas the assessment about whether it was right or wrong can only be made with the benefit of hindsight.

Take the example of two teens, Brian and James, both of whom attended a late‐night party where alcohol was consumed. Brian elected to get a ride home from a friend who had been drinking heavily. James chose to ride with a friend who hadn't had a single drink. Brian made it home without incident, whereas James was paralyzed when a distracted driver ran a red light. It's clear to anyone reading this that James had made the rational choice, whereas Brian made the right one. I know it's hard to accept that the correct term to use for Brian's choice is right, but according to the definition, it is.

There is no controversy in a situation like this. James made the correct choice even though it worked out poorly. Even knowing the outcomes, you would still say that James made the rational decision whereas Brian had been foolish. Unfortunately, few decisions are as cut and dried as this. More often than not, we are not accurate historians of decisions, but rather revisionists. Once the outcome is known, the initial expectations are erased forever. Decisions that were once deemed rational are recategorized as suboptimal simply because they didn't turn out to be right. Information that led us astray is discounted, whereas that which corroborates the outcome is elevated in value, and the “mistakes” are now glaringly obvious.

It isn't intentional; it's just what our brains do to help us make sense of a complex world. However, in the case of James and Brian, it's simply impossible to retrospectively convince ourselves that Brian decided well and James poorly, so we make sense of it in one of three ways. We question whether the sober teen was in fact sober, assume there is more to the story, or assign it to a higher power. When it comes to the decisions we make as investors, however, more often than not, we play the role of revisionist historians, and that doesn't come without consequences.

The technical term for what I've described is hindsight or outcome bias. It's when you change your assessment of whether a decision was rational simply because you know the outcome. Here's where it gets tricky. You gather copious amounts of historical data, speak to numerous experts, listen to opposing viewpoints, and develop a rational strategy given all the relevant information. Your boss (or investors) agrees that your analysis is sound and your expectations are reasonable. That's not to say that they don't see how it could potentially go wrong or that they are unwilling to express those concerns in the moment. Of course, there are always risks, and you've properly considered them. In light of the risks and rewards, given their appropriate weightings, you all agree the investment is warranted.

Unfortunately, you aren't judged by whether your decisions are rational. Your decisions, your investments, your portfolio, your fund, and potentially even your own self‐worth are more often judged by whether they were right. In other words, your decisions are judged by the outcome, not the process, and that can be dangerous for a number of reasons.

The table in Figure 3.1 was posted on LinkedIn. No caption was attached. No mention of the fact that it was based on only three data points, nor the decision‐making that led to the results. But that's what we do. If a decision turns out to be right, it must have been due to solid decision‐making, and vice versa.

Well, if Brian, the teenager who rode home with a drunk driver were a fund manager, he'd have been among the top performers, whereas James would have been in the bottom. The thing is, correct (or right) decisions can only be judged with the benefit of hindsight, but investors have to make investments and allocate to managers prior to the results being known. Therefore, as investors, we must somehow make the distinction between returns that were generated by (1) rational decisions as part of a consistent decision‐making process, (2) a selection of rational decisions that are part of an inconsistent decision‐making process, or (3) simply irrational decisions that happened to work out well.

Just because Brian wound up getting home safely doesn't mean that we should have expected it, nor does it mean that we should expect the same result going forward. However, since his poor decision wasn't punished, we should expect that he will likely employ that same decision‐making process in the future. Given that likelihood, Brian isn't someone I would expect to survive very long.

As it relates to investment management, there are two distinct difficulties in conducting a proper assessment, and this applies not just to the allocator, but to the traders and portfolio managers themselves. First, once you see the result, it's nearly impossible to unsee it. When the outlier event that you had acknowledged beforehand actually comes true, the immediate response is, “I knew it!” However, the truth is, you didn't know it. You simply considered it to be a possibility, but with the benefit of hindsight, that outlier possibility appears to have been all but inevitable, not just to the outsider judging them, but to the decision‐makers themselves. The same goes for decisions that work out well. When the stock markets rally, investors often ask, “Why should I invest with a hedge fund and pay 2/20, when I could get that kind of return just being long the S&P?” That comment, as well as the inability to acknowledge that when a hedge fund manager goes long equities the resulting returns are alpha both serve as evidence of hindsight bias. I'll expand on this shortly.

The only true way to overcome this bias when assessing a manager, or your own trades, is to keep records of the decision‐making process, complete with all the evidence gathered, the probabilities assigned to different outcomes, the drivers for those probabilities, and the triggers for meeting/missing those expectations. In so doing, a trader who has a cold streak or who misses an opportunity they had once considered, can maintain a state of equanimity. That in turn improves the odds that they will continue to make decisions that are probabilistic in nature, rather than emotionally driven by P&L. It also provides evidence of rational decisions being made as part of a consistent decision‐making process. That's important because it means it is repeatable.

Here's the problem with having decisions judged by the outcome. If my boss and I agree that I am making a rational decision by investing in XYZ and it doesn't work out, but he maintains his opinion of me based on the fact that my decision was rational, then I will continue to make decisions in the same manner. If, however, in light of the outcome he changes his opinion of me, then he has just injected uncertainty into my decision‐making process – an uncertainty that cannot be resolved. That tends to lead to a more bureaucratic and risk‐averse approach. I will explain why this is the case later.

Going back to our teens, how would either of them adjust their behavior if Brian, having arrived home safely, is rewarded by his parents for making the right decision, whereas James is punished for having been in an accident? Of course it sounds silly, but how is it different from what is done by allocators and traders every day?

The truth is that rational decisions provide the best odds of success over time. If I want to lose weight, I will reduce the number of calories I take in and increase the number of calories I burn. In doing so, I improve the odds that I will lose weight. Does it mean I will lose weight every day, every week, or even every month? No. Is it possible I can gain weight at some point along the way? Yes. Should I change my approach in those moments? Of course not.

There are kids who drive drunk without incidence for a time. There are people who chew tobacco for many years without any problems. There are those who can eat whatever they want without gaining weight while they're young. There are investment managers who make irrational decisions or lack a consistent investment process of any kind, yet generate positive returns for a while. However, none of these represent good bets over time.

Right is not the same as rational. Rational decisions simply improve our odds of being right. If rational decision‐making is repeated in a consistent manner often enough, you should expect to be more successful than someone who does not. Is it guaranteed? No, but we are in the business of playing the odds. It's true, past results do not necessarily guarantee future performance. However, rational decision‐making is certainly more predictive of success than those that have simply been right for a time. There will be times when we will be right and times when we will be wrong. All we really control is whether we are being rational. The question is, how can we tell if we're being rational?

Swing Analysis for Portfolio Managers

In 2013 the Seattle Seahawks lost in the divisional‐round of the playoffs. In the press conference that followed, Russell Wilson, the team's quarterback and leader on the field, talked repeatedly about how he was looking forward to analyzing the game tapes. Now that may not sound odd to you, perhaps because you've heard that sort of thing so often, or maybe because it just makes sense. Take a second to think about it for a moment, though. At his fingertips, Russell has every stat imaginable readily available to him, including the final score, pass completions, completions by receiver, batted passes, yards per catch, yards after the catch, and on and on. There isn't any statistic you can think of that he can't access at a moment's notice. Why then would he take the time to review game film?

When a reporter asked exactly that, here was Russell's answer.

I like to watch my footwork more than anything and see, you know, am I getting the ball out on time? Are my reads correct? Am I reading it the way I wanna read it in terms of the progressions? I think for me, when I'm getting my depth and my drops, and I'm making the right protection calls, and I'm getting back in my drops and just trusting the right read, it's pretty hard to stop. I think that's what shows on the film. I just basically play the game over again in my head. I watch all the down and distances, and look at the situations again, and just play the game again.

Russell went on to lead the Seahawks to a stunning defeat of the Denver Broncos and quarterback Peyton Manning in the Super Bowl the very next season. Denver quarterback's coach, Greg Knapp, spoke about his conversation with Peyton Manning after the game. “I told him the day after the game we were going to watch it when he came back, not right then, but the first day he was back, and we were going to watch it without the emotion of what just happened.”

Opening the wounds of the Super Bowl was only the first step of a complete deconstruction of the previous season – a pass‐by‐pass analysis by Manning of Manning. For one of the most meticulous minds in the game, it was the beginning of weeks and months spent breaking down not just opposing defenses but himself.

Manning judged his decisions, footwork, and throwing motion one pass at a time. He did so not from the overconfident perspective of one of the greatest to ever play the game, but rather as someone eager to uncover and learn from his own mistakes.

Manning put himself under the microscope.

“If you ever feel like that's not important – like, ‘Hey, I don't need to watch last season; I know what we did; I know what I did wrong' – no, you don't know,” Manning said. “You need to watch it. Watch the bad plays. It's not fun to watch bad plays, to sit there and say, ‘That's a bad decision' and ‘That's a really bad decision' and ‘Horrible read.’ …No matter how old you are, you need to go into that prepared to be constructively criticized and learn how to grow out of the mistakes every year.”

The value of video analysis in sports isn't really in question. Odds are if you've taken a golf lesson in the past five years that video analysis of your swing was probably a big part of it. In fact, golf instructors say most clients expect, and often demand it. The same goes for baseball players, tennis players, runners, and even swimmers.

Years ago, it dawned on me that I, too, could benefit from a similar process, wherein I reviewed not just my P&L and trade statistics, but an objective break down of every play, every progression, and every read; my play calling and footwork. In other words, rather than reviewing the outcomes, I would review the process itself. But how? We can't film the thought process that leads to a trade. We don't record our thoughts when we change our minds midway through the life of a trade. It isn't possible to see whether it was a bad view, poor structuring, or pressure from management at a moment of psychological vulnerability that caused a position to falter. Or is it?

By chronicling every trade, including why it is on, what the expectations are for every factor I see influencing it (and me) over time, what my plan of action is, reasons it's unlikely to work out in the end, and updating it all as the trade progresses, I am effectively creating game film that can be reviewed and critiqued, objectively and without emotion. Particularly when I'm experiencing a period of poor performance, I can remain unbiased, to continue seeing things clearly so that I don't compound a bad turn of events with bad decisions. Whereas many are likely to ease up on the critical analysis when they're doing well, it is equally important to recognize when good luck is raining down on you, to remain alert, disciplined, and yes, unbiased in that very vulnerable moment.

One of the other benefits of video analysis in the sporting world is identifying tendencies that are likely to result in injury over time. Without a thorough analysis of those potentially damaging tendencies, athletes, particularly those with a strong work ethic, who tirelessly practice their swing, over and over again, are at great risk. The same could be said for professional investors. What tendencies do you have that, in times of stress, you spend more time on that may actually put you at greater risk?

Asking the Right Questions

Richard Cantrell is a 48‐year‐old institutional investment manager. His teenage daughter, Jessica, is going to a concert with her friends tonight. They are a large group so will be traveling in two cars, one driven by 19‐year‐old Todd and the other by his 16‐year‐old brother, Bobby. Naturally, Richard is concerned about his daughter's safety, so he wants to be sure she will be in the car with the better driver. Luckily, the other kids are being dropped off at Richard's house and the whole group will leave from there, giving him an opportunity to have a conversation with the brothers before she leaves.

Todd is home from college for the weekend. He's a freshman studying finance at Richard's alma mater and even joined his old fraternity. He's driving a five‐year‐old Volvo that looks like it just came off the showroom floor. His younger brother, Bobby, isn't sure which college he wants to attend. At the moment he's more concerned about his new Ford Mustang, which is going to the body shop tomorrow for repairs. Apparently, “some old lady” wasn't paying attention when she backed out of her parking spot and clipped the back corner of his car. Neither of the boys has received a traffic violation, or at least neither was willing to admit it to Richard. They are both clean cut, well spoken and would never think of drinking and driving. If it came to it, which “it most certainly will not,” they would call their father to pick them all up rather than drive while intoxicated.

Based on the information gleaned during his investigation, with whom would you have wanted your daughter to ride?

Well, Todd is older, making him a more experienced driver, not to mention he drives a car commonly associated with safety, which also happens to be in pristine condition. He is following the same life path as Richard, so naturally he assumes Todd is likely to approach other decisions in a way that is consistent with his own as well. Although it's possible young Bobby is telling the truth about the old lady, Richard has yet to meet a teenager who ever thought anything was their fault, so he treats Bobby's story with a grain of salt. Plus, Mustangs are muscle cars and we all know what type of driver is attracted to that kind of power. Richard is the analytical type. He doesn't like to rush to judgment, but given the information available he would rather be safe than sorry, so despite her protests, he insists that his daughter ride with Todd, or not at all. She promised to abide by his wish.

Jessica kept her promise, but never made it home. On the way back from the concert, Todd ran a red light. An 18‐wheeler plowed into the passenger side of the Mustang, killing all four occupants. Todd's blood alcohol level was nearly twice the legal limit.

Richard consoled himself with the knowledge that he had done all he could to keep his daughter safe. Given the information available, he had made the right decision. The problem is, Richard had fallen prey to numerous cognitive biases, which, despite his best intentions and experience, undermined the entire decision‐making process and led to a fatal error. Let's analyze Richard's assessment to see what I mean.

Roughly 50,000 students from all walks of life currently attend Richard's alma mater, and hundreds of thousands more attend schools that are nearly identical to it. In the years since Richard's graduation, millions have passed through that institution and others like it. To assume Todd approached decisions in a similar way to Richard solely because of the institution he attended is farfetched, no matter how much it appeals to his, and our, intuition. Todd was home for vacation from school where his battered Camaro was parked that night. His parents, knowing James' poor driving record, insisted that he drive their Volvo while he was home. That didn't sit well with James and so after “one or two beers” at the concert, he pressured his younger brother to swap cars for the ride home.

While Todd was sincere when he told Richard that he wouldn't drive while intoxicated, unfortunately, Todd had a different definition of what it meant to be intoxicated. He truly believed he was “okay to drive.” In fact, like Richard, Todd also believed he was a better driver as a result of his three years behind the wheel (see Figure 3.2). His overconfidence resulted in a more cavalier attitude toward the driving experience. So, unlike his younger brother who was still a cautious driver, James liked to drive with the music turned up loud and didn't think twice about other potential distractions caused by his passengers. Todd truly believed he was a good, responsible driver who was prepared for any situation. After all, he'd been driving successfully for three years without a single fatality or even fender bender (that he considered to be his fault), and he was responsible because he always demanded that his passengers wear their seatbelts.

The decision‐making processes of both Todd and Richard are representative of the process so pervasive in our industry, complete with all the usual cognitive biases, including framing, representativeness, the halo effect, and availability. Consider the following.

Frank Swanson is the Chief Investment Officer of a large endowment. He is a stickler for details. Before he will consider investing in a particular hedge fund, the fund must pass a screening process based on a checklist of items that, over the course of many decades, have become generally accepted characteristics of the most successful investment managers. In addition, he relies on the intuition honed over his 25 years in the business. If something just doesn't feel right to him, he will pass. It's a process that has served him well, as evidenced by the fact that he now oversees $45 billion in diversified assets.

Today, Frank and his team are meeting with the leaders of two young hedge funds. The first, started by the former global head of foreign‐exchange and interest‐rate trading at a marquee investment firm where he had a reputation for making a lot of money, launched with $500 million in assets under management (AUM); $50 million of it came from his own pocket. With three years under their belt, they haven't knocked it out of the park, nor have they blown up. Like most established firms, they've experienced a few good runs with the occasional rough patch, but nothing that should raise alarm bells. They have 15 analysts, an economist who used to work at the Federal Reserve and another from the Bank of England, plus 10 portfolio managers with experience in a wide range of asset classes. They employ tight speed bumps to ensure no single portfolio manager can “do too much damage,” and in any case, the majority of risk is taken by the chief investment officer (CIO). They use the most revered law firm, accountants, auditors, and fund administrators to handle their administrative and operational needs, and employ the most trusted risk‐management software, which is overseen by an independent risk officer. The firm subscribes to the best‐known independent research and has direct access to thought leaders throughout the industry.

A man who had been a portfolio manager at three different hedge funds over the past 10 years runs the other fund being assessed today. All three had failed for three different reasons, none of which appeared to be tied to his actions. Although he had produced excellent returns, his Sharpe ratio was barely above 1.0. Although he had run several very profitable institutional trading businesses prior to moving to the buy‐side, they were all within second‐tier banks and insurance companies, so he isn't well known throughout the industry. The firms who handle their legal, accounting, auditing, and administrative work are also well known and reputable. The CIO doesn't believe in outsourcing research, trade structuring, or execution, so he doesn't have any analysts, nor do they subscribe to outside research either. All ideas are internally generated by scouring economic and market data in the same manner that produced his top‐tier track record, and goes a long way to explain why his returns are so uncorrelated to every major index. His fund currently has just $20 million in AUM, mostly from friends and family, but he has previously managed upward of $1 billion in his independent portfolios.

Based on the information gleaned during Frank's investigation, with whom would you have invested?

If you are being honest, the clear winner is the first fund. Purely from a marketing and fundraising perspective, simply coming out of a prestigious firm, such as Goldman Sachs or Morgan Stanley, provides a distinct competitive advantage. That credibility translates into commitments from other credible institutions, which creates a credibility cascade, resulting in the $500 million launch. With such a base, other large institutions with both minimum investment and maximum proportion rules in place can participate. Being familiar with the allocator “checklist,” the first manager catered to them all. Truth be told, he believed in the value of the checklist, too, which is why he didn't hesitate to pull together so many analysts, economists, and advisors.

As for the second fund manager, while he may have a more transparent and superior track record, qualifying his accomplishments requires more work. He suffers from something akin to guilt by association. Although it is clear that he was not responsible for the downfall of the firms at which he had been employed, simply being associated with them created a mental hurdle that had to be overcome, precisely the opposite of what the other manager had working in his favor, at least on the surface. In reality, they were both affected by what is known as the availability cascade.

It is a phenomenon in which one bit of information creates a baseline expectation and each additional bit serves to reinforce that expectation. Our ears perk up to new information that supports the original narrative, while contradictory evidence goes unnoticed. Every step of the way, the story seems to unfold, becoming more powerful and seemingly more obvious, regardless of the predictive value of the data, or even whether it is valid.

Mistakes in Common

Truth is, in both Richard (the dad deciding which car to have his daughter ride in) and Frank's (The CIO deciding how to allocate assets) analyses, there wasn't any information provided that was of any predictive value. Every bit was meant to trigger bias in you, the reader, by appealing to our deep‐seated beliefs. Even the description of Frank conveyed little information of real value, yet it likely generated serious bias within the reader's mind.

The both analyses were perfunctory at best. In neither case was any inquiry made regarding the decision‐making process of any of the participants. How did Todd arrive at his decision to attend Richard's alma mater? Why did the first manager employ tight speed bumps? How did the second manager deliver such uncorrelated returns so consistently? How does Todd define intoxication in that moment? Would one beer qualify? Why did he select the Volvo? How much did the first manager's group at his old employer make in the five years before he took over relative to what it made under his guidance? How about the three years since? Why would you (not) employ analysts? Questions that should have been asked were ignored, and in the absence of those answers, the missing pieces were filled in with bias.

When you understand how decisions should be made in order to generate better results over time, you have a better understanding of the questions that should be asked in order to better predict the performance of drivers, traders, portfolio managers, CIOs, and allocators. To do so requires fighting our natural inclination to achieve cognitive ease by relying on gut feel and intuition, in favor of inviting cognitive strain by asking questions that require deep reflection and investigation. Truth be told, the correlation between alpha generation and the information being used to make investment and allocation decisions by the majority of decision makers in this industry is highly speculative at best. Simply working at a highly regarded, very profitable investment firm is no more predictive of the ability to generate alpha as a hedge fund manager than attending your alma mater can predict a teenager's ability to deliver your daughter home safely. More on this shortly.

Descriptive versus Prescriptive Decision Theory

The second of the two dominant disciplines in the study of decision‐making is descriptive decision theory. Whereas normative decision theorists study how we should make decisions, descriptive decision theorists study how we actually make them. For the past several decades, a growing number of researchers have been focusing their efforts in this area. Some of them, like Daniel Kahneman, have developed something of a cult following for their work, and rightly so. What they've discovered is that far more often than any of us would like to believe we make decisions in a way that differs from how we should be making them.

I am involved in a third, less‐traveled discipline within the field, known as prescriptive decision theory. Here, we attempt to help decision‐makers close the gap between how they actually make decisions and how they should. In order to do so, we must first understand what it is that exists in that gap. Simply stated, what exists between how a decision is made and how it should be is a mistake (Figure 3.3). Remove the mistake and you have a decision made in the manner in which it should be.

The question we should be asking then is, why do we make decision‐related mistakes, and is there a common theme among them? The answer has a lot to do with the sheer volume of decisions that we face on a regular basis. It simply isn't possible to thoughtfully collect data, then properly process and assess it in order to select the most appropriate action for every one of the million plus decisions we face each and every day. In case you think that figure represents an exaggeration, consider how your day begins.

The alarm goes off. Before you open your eyes, you choose whether to hit snooze. Then you repeat the decision a few minutes later. At some point, you decide to get out of bed. Then, should you relieve your bowel? In the toilet? Sitting or standing? Lift the seat? Flush when you're through? Lower the seat? Wash your hands? Hot water, cold, or some combination thereof? How much soap? Rinse? Brush teeth? Hold it in left hand or right? Put toothpaste on the bristles only, or across your knuckles? For how long should you brush? How long on each tooth? Leave the water running while brushing? Rinse? Shower? Breakfast? You get the point. Before you have even removed the sleepers from your eyes, you have already faced and made hundreds of decisions. Effectively, hundreds of decisions made before you are even awake. Choices that affect the rest of your day and your long‐term health, for all intents and purposes, made routinely without you being involved as an active participant. Instead, you make many of these decisions through a combination of habits, intuition, and gut feel. They are mental shortcuts, or what cognitive scientists call heuristics, used to make so many of our more automated decisions.

Most scientists would agree, employing heuristics is absolutely necessary. Our brains just don't possess the processing power, nor do we have the time to apply a proper decision analysis to every choice we face. For the majority of these routine tasks, making decisions by employing heuristics works just fine. Theoretically, heuristics allow us to be more efficient decision‐makers, freeing us to focus our attention and employ cognitive strain where the consequences are significant. Unfortunately, heuristics are so readily available and require so little effort to employ that we tend to rely on them far more often than we should even when the consequences are as significant as life and death. It is in the moments when we should be employing cognitive strain, but fall back on heuristics, that we make the majority of our mistakes. Although heuristics are necessary for us to function on a daily basis, when they are employed in situations that require a more thorough analysis, we become vulnerable to unconscious influences that have a tendency to produce systematic errors in judgment.