Should Risk Management Be Based on Process or Judgment?
We can make this chapter real short and say “it depends” to the question of whether risk management should be based on process or judgment. The problem is that, for most companies, risk management process is not set up to be quite that flexible, or that ambiguous. For a combination of reasons, mainly legal, regulatory, and insurance reasons, most companies have their risk management based almost exclusively on process, with little allowance for judgment. So much for the obvious answer of “it depends,” but I guess it means that I need to continue writing some more in this chapter to explain myself.
Tomas Lopez
Let’s begin our discussion with the real-life case study of Tomas Lopez. You may recall hearing about him from the news a few years ago when he had his 15 minutes of fame. Tomas Lopez was a 21-year-old lifeguard in the city of Hallandale Beach, which is in South Florida. The instructions for Tomas were very simple and very straightforward; if someone is within the roped off swimming area and in distress, go save them; if they are outside of the roped off area, call 911 and wait for them to arrive and have the 911 responders save the person. Very clear and unambiguous instructions, but for some reason, Tomas failed to follow the script. Someone who was swimming outside of the designated swimming area got into distress in the water and Tomas went into the nonroped off area and saved them. He was fired for not following the process.
If you are a risk manager, you likely believe that Tomas screwed up. He did not follow the process, and if he was not fired, then it is likely that he would have been made to attend a mandatory risk management refresher training program for his lack of risk management knowledge. If you were the person in swimming distress, then you likely have a different view on the proper action for Tomas to take. (If you were the person in swimming distress, you also have some explaining to do to account for why you were not following the rules and were swimming outside of the authorized area.)
This may appear to be a rather unusual situation—I suspect that there are few managers reading this book who are responsible for lifeguards—but the questions it raises about risk management are all too common. Ignoring the moral implications of the decision for a moment, there are excellent reasons for the policy that was in place. First, if the policy about not rescuing people outside of the roped off area is not enforced, then people will naturally enter into the water wherever they desire, which, in turn, will make the job of the lifeguard much more difficult, as they need to keep track of people over a much larger area, and thus, dramatically decrease their ability to effectively do their job. It will encourage swimming by oneself, and thus, cut down on the number of people employing the buddy system. It will also greatly increase the insurance liabilities and the potential lawsuits.
There is also the issue of what would have happened if there was an emergency with someone who was swimming in the appropriately roped off area while Tomas was saving the rule-busting swimmer. Obviously, all legal heck would have broken loose if a rule-abiding swimmer got into trouble and was not able to be saved by the diverted Tomas. (In this actual situation, other lifeguards made themselves available to cover the roped off area that Tomas abandoned.) One can easily imagine the media talking about the irresponsible lifeguard who broke procedure and saved a rule-ignoring anarchist while allowing a rule-loving patriot to perish. The letters to the editor page would have been filled with angry letters about how the fates of the two swimmers should have been reversed, and there would be follow-up stories about how Tomas had to change his identity to escape the wrath of the public that he and his family faced.
A strong argument can be made either way for the procedure that was in place for the lifeguards to follow. Having a process-based procedure helps the lifeguards—who tend to be young and without much experience in making life and death decisions—in doing their important risk management task in the best way possible. It certainly helps for legal and insurance reasons, and it allows for the critical decisions that need to be made in an instant to be deliberated at length in an atmosphere and a timeframe that is more conducive to properly thinking through all of the implications. A split-second life or death panic situation is not the best context for making decisions (or is it?).
On the other hand, the policy takes a lot of decision out of the hands of the actual employee who in the moment has the responsibility for the decision—in this case, Tomas. Some people, particularly those who want little to no responsibility whatsoever, believe that having a hard- and-fast procedure to follow is a real blessing. Indeed, some companies want employees who want that; for example, franchisers who want each of their outlets to be operated in the exact same manner. These employers do not want employees who think; they want employees who follow rules and processes unquestioningly. However, for most modern jobs, you want employees and staff who have a brain, and having a brain means people have the capability for, and a desire for, at least some accountability.
The Tomas Lopez situation also quite adequately brings home the issue of risk tolerance. I suspect few organizations have the tolerance to have their process take precedent over a preventable death. This brings up serious questions of when obedience to a process should be broken. There are clearly moral and ethical implications that go well beyond risk management.
There is also the possibility for a debate about the usefulness of Bayesian risk analysis. There is a known swimmer in distress versus the relatively low probability of a swimmer in distress in the roped off area. On the probability of saving a life, the analysis appears to be clear.
Ethical issues, insurance, risk management protocol, probability of saving a life, keeping one’s job—it is a lot for a young lifeguard to have to process in real time. A well-defined and understood process makes the decision of the lifeguard much easier in the moment, but is it the right decision and what are the long-term implications? Process versus judgment is a very wicked problem.
To conclude the story of Tomas, he was fired from his lifeguarding job by the contractor who hired him, along with several other lifeguards who were involved in the rescue. The reason for the firing was that Tomas left his post to save someone in the unauthorized swimming area. The media backlash was both swift and harsh. The city and the contractor were soundly criticized, and as a result of the backlash, Tomas was offered his job back—although he refused to accept the offer to return to lifeguarding. Tomas received the key to the city for his quick decision making and was praised as a hero.
Processes and Obedience
It is likely that you think that Tomas Lopez made the right decision; or maybe you think he was wrong. I am not going to judge your opinion either way. The point of the case is to highlight the central problem behind process versus judgment. The whole reason for developing a process for risk management purposes is that you do not want any deviations from the prescribed protocol. The flip side is that it is virtually impossible to design a protocol that will cover all conceivable risk situations, and you need people to be able to use appropriate judgment for the situation at hand.
Regardless of what you believe that Tomas should have done, it is virtually certain that there will be times when you do not want the process to be followed. You might think it will be obvious when it is prudent to not follow the rules. You might believe that if such a unique and rare set of circumstances occurs that any reasonably rational employee will both know to not to follow the standard procedure, and will have the intelligence, and the will to break protocol, and will do what needs to be done. That is a lot of “ands” and a lot of wishful thinking—particularly, if the risk management function has been set up to be rules-based versus judgment-based.
Despite the fact that you probably illegally jay-walked several times today (or did some other similar trivial illegal activity), the reality is that most people are law-abiding and will try to the best of their ability to follow rules as they are presented to them. (Teenagers versus parent’s rules being the obvious exception to this, but that is a subject for a different book.) So, in the workplace, you might think that if there is an obvious rationale to temporarily ignore the risk management rules that employees will do so. However, the famous experiments of psychologist Stanley Milgram show that is not the case and why it is critical to think through the issues at hand before instituting a rules-based process.
In case you forgot your freshman psychology class, Stanley Milgram did a series of experiments to show how far people were willing to go against their better judgment and actually torture people in order to follow orders from an authority figure. In his famous experiments, Milgram had an authority figure who interacted with two research subjects, although one of the “research subjects” was actually an actor and a known part of the experiment. The subjects were told that the purpose of the study was to examine how receiving electrical shocks would affect learning ability. The “true” research subject was to ask the “actor” questions, and whenever they got an answer wrong, they would be given a shock. As the number of questions answered incorrectly increased, the shock would become larger and more intense. The “true” research subject was given a relatively mild shock at the inception of the experiment, so that they would have some appreciation for what the “actor” was feeling.
The “true” research subject was placed in a room, and in an adjoining room, the “actor” was supposedly hooked up to some electrodes. The research subjects could not see each other, but they could hear each other. As the experiment progressed, the “actor” would get more questions wrong and the “true” test subject would have to move a dial to increase the size of the shock they would receive. In reality, of course, the “actor” was not really hooked up to any electrodes, and was not receiving any shocks, but the “true” test subject did not realize this. As the shock increased, the “actor” would start crying out in pain, and eventually start begging the “true” research subject to stop. The authority figure, however, would remind the “true” test subject that they had agreed to participate in the experiment, that they were being paid to do so, and that it was important for the scientific research about learning.
The experiment, of course, was not to research learning abilities but was to see how far people would go in behaving a set of rules that were put in place by an authority figure. The results were disturbing at the time, and they remain disturbing to this day as the experiment has been replicated in many different forms. What Milgram found was that a large majority of the test subjects would continue to give larger and larger “shocks” to the “actor” even though they knew it was wrong to do so and even though they themselves were extremely stressed about doing so. In fact, many “true” test subjects continued to increase the size of the shock even after it appeared that the “actor” may have been killed by the shocks. When test subjects wanted to stop the experiment, the authority figure would remind them of the rules and that they had agreed to participate according to the rules.
The implications of this for risk management and process-based risk management are straightforward. If given a set of rules or processes to follow, and if the processes or rules are thought to be coming from an authority figure, then the reality is that people will be very reluctant to deviate from those processes and rules, no matter how much they believe that they should. Tomas Lopez was an outlier. As a risk manager, you cannot assume that people will use their judgment if a rule or a process is in place, despite how obvious it is that they should abandon the rule for that specific instance and use their judgment instead. If you are going to put in process-based risk management, you had better make darn sure that it is exactly what you want people to do, and that it covers all conceivable situations.
Simple, Complicated, or Complex (Again)
In Chapter 3, I mentioned the best-selling book by Atul Gawande titled The Checklist Manifesto: How to Get Things Right was discussed.1 In dealing with situations that are simple, a checklist—or a process if you will—proves to be a very powerful tool or tactic for risk management purposes. A checklist, no matter how simple, or no matter how obvious the steps may be, helps to avoid mistakes and ensure that opportunities are captured. (It also deadens the brain for risk management, but this will be discussed more at length in Chapter 9: Can Your Risk System Be Too Good?)
Checklists for simple systems work because simple systems more or less follow rules and because they are robust. A slight to medium alteration to the rules will still produce an acceptable outcome. Complicated systems, which adhere strictly to rules, are also amenable to risk management by processes, and in fact, complicated issues are the ideal type of situation that should be managed by a process or even by an expert system such as a computer.
More and more we see processes being used to risk manage complicated systems. In fact, we are going to extremes to manage these systems using processes. Because complicated systems are completely reproducible, do the exact same thing and get the exact same outcome, and as they strictly adhere to a set of rules or regulations, they can, in essence, be managed by processes that can be digitized. In other words, for complicated tasks, we can program a robot or a computer to manage them. After all, nothing follows rules more unquestionably than a computer. Problem solved! Computer-driven automobiles will save lives and likely get us to our destinations more quickly and in a more fuel-efficient manner. Automated security systems have replaced security guards at the entrances to the most secure parts of an organization. Computer systems might be hacked, but that is generally more difficult than bribing a guard or overcoming a guard by force. Robots are used in combat and in police work for the most dangerous of situations, and drones are used where it is too risky to fly manned surveillance aircraft. In many instances, the only use of humans in risk management is to give comfort to the general public who do not fully understand, or more likely, do not want to understand how much more efficiently and effectively that bots can manage most complicated tasks.
Complicated systems are so efficiently managed by bots that we create a new problem; that of trying to make every risk management issue a complicated one. We do this by assumption, or we do this by simplifying the situation. Sometimes, it is done by making heroic academic assumptions.
To take a basic example, we often assume that financial market returns are normally distributed. This allows the power of the well-developed mathematics of the normal distribution to be applied. Although we are still left with probabilistic outcomes, and not the completely reproducible outcomes of a true complicated system, it, at least, gives the appearance of such. Thus, we build pricing models and we value financial market securities with a false sense of great precision and accuracy. It then produces shock and surprise when a financial institution(s) suffers a large unexpected loss. A computer almost certainly produced a price, based on a complicated systems model, and no human checked to see whether the answer or the algorithm made intuitive sense as the mathematics were so elegant and academically verified. Regulators then set capital standards based on the models, and then later add a buffer factor to account for the unknowns. Processes- and rules-based risk managements do not do so well with unknown unknowns.
The issue with risk issues that are complex is that solutions are not possible. You cannot devise a process, an algorithm, or a set of rules to manage something that is complex and exhibits the property of emergence. As discussed in Chapter 3, the way to manage complexity is to think in terms of “manage, not solve,” and also to think in terms of “try, learn, and adapt.” There is no reproducibility in complex situations, and thus, there is no consistent way to manage or react to complex situations. Each instance is unique and inherently context-specific. Complex systems also tend to exhibit high levels of nonlinearity that produced very different outcomes based on very subtle differences. This is often referred to as the butterfly effect: the fact that a butterfly flaps its wings on the west coast can change the weather from sunny to a thunderstorm on the east coast a few days later.
Complexity requires judgment. You cannot digitize or codify responses to complexity. Complex situations require a manager who has the experience, the wisdom, and the courage to make judgments and the creativity and tenacity to continue to adjust their decisions. It requires real-time, context-specific management.
Risk situations that are complicated, and those that are complex, form the boundaries for the process versus judgment debate. To the extent that an organization has risk issues that are complicated, then they should utilize processes as their main risk tool. Conversely, those organizations that operate with mainly complex risk issues will require judgment as their main mode of risk management.
The practical reality is that for most of the time, risks will behave as if they are complicated. That is, they can be effectively managed by rules, regulations, or processes. The tricky thing is that occasionally, these seemingly complicated risks will take on patterns of complex risks. Then, a switch in management techniques needs to be made. Knowing when to make this switch from complicated thinking, and thus, process-based management, to complex thinking and judgment-based management is, perhaps, one of the most critical, yet, one of the rarest characteristics in effective risk management. It is doubtful if this trait can be taught, and it is doubtful if it is a characteristic that can be screened for in a hiring interview. It would seem that it is a trait that is possessed only by those who are highly intelligent and have the ability to live in the moment, although this assertion can be easily questioned and challenged. What is clear is that Tomas Lopez is one of the few who possess this ability to switch modes—or at least he did on the day that he performed his life-saving act of judgment.
The dichotomy between risks that are complicated and those that are complex illustrates the need to be able to understand and recognize the difference between complicated and complex situations. In terms of being able to effectively manage risk, it is unfortunate that so few important risks are complicated. On the other hand, in terms of providing rewarding careers for risk managers, it is fortunate that so many important risks are complex. Without complexity, risk managers could be (and should be) replaced by processes managed by robots and computers.
Playbooks Not Rulebooks
Jeff Swystun, President and Chief Marketing Officer of Swystun Communications, has a philosophy for creating great advertising that I believe also applies to risk management. That philosophy is “playbooks, not rulebooks.” It is a great phrase to use to encapsulate the discussion about whether or not risk should be process or judgment-based. A playbook is a set of guidelines that should be roughly adhered to whenever practical. However, they are not rules that must be followed. When it is not practical to follow the guidelines, or when intelligent intuition says that a different path should be attempted, then the playbook takes a secondary role to responding based on the context of the moment.
I liken “playbooks not rulebooks,” to coaching a team sport such as hockey. Before the game, and during a timeout, you will frequently see the coach with a whiteboard explaining how she or he wants the team to line-up and proceed with a tactical play or plan. Of course, the other team’s coach is also doing the same thing and obviously both teams cannot, and will not, get their actual realized play to correspond to their respective plans. In fact, each team is doing their best to annul the other team’s plan. Thus, each team goes back into action and implements the plan as best as they can, but it is likely very different from what the coach drew up. The team that is best at improvising and adjusting the plan according to some very basic principles is the team that generally succeeds.
Consider what would happen in the silly situation where a team is not able to follow the first step of their coach’s plan, and thus, stops playing while the other team continues to play on while improvising. Obviously, the team that can only play to the exact specifications of their coach’s plan is not going to do very well at all. In the early days of primitive computer chess programs, one could easily beat the computer by making highly unusual moves early in the chess game. For instance, moving the rook’s pawn as one’s first move would render many early computer chess programs useless as they could not compute how to react against such a strange move.
Playbooks, not rulebooks is, of course, a compromise position in the process versus judgment debate. Compromise solutions generally do not go over very well when asked to provide a solution or an answer. However, in our increasingly complex world, compromise solutions are often the best. Although board members, regulators, and stakeholders want assurances and documented proof of solutions, the reality is that business is not as black and white as we would like it to be. For some situations, a process works very well. For other situations, we have to fall back on judgment. At least, it makes risk management interesting.
Hiring Strategy
Before concluding this chapter, it is useful to take a few minutes to consider the hiring practices of an organization. No matter where an organization falls on the process versus judgment debate, its hiring practices should be consistent with it. If a company has a rigid process-based risk management system, then the people it hires should be amenable to such a system. One company I know of, who for obvious reasons will remain nameless, likes to hire people with low-level military experience, as they assume they are accustomed to following orders without question. If a company has a more judgment-based risk management process, then the people it hires should have the intelligence, maturity, and intuition to act on that judgment. The risk mindset of the hires needs to properly align with the risk system employed.
The second component to this is that the risk management training also has to be consistent with the style of risk management. If the risk system is process-based, then training needs to convey what the processes are, and perhaps, more importantly, why the processes are in place. When risk management is process-based, it is often incorrectly assumed that all one needs to know is the rules of the process. However, compliance will increase dramatically if the employees also know the why. Understanding the consequences—both good and bad—of compliance and of noncompliance helps the users who need to implement the processes appreciate the effects of their action. When someone understands why they are to follow a process, then they are much more likely to do so. They are also much more likely to appropriately deviate from the process when special circumstances dictate that they should do so.
Training for judgment is much more difficult than training someone how to operate a process. Often, it is easier to hire for judgment than to train for judgment. After all, one can only assess one’s judgment with experience, and likewise, experience is the best trainer for judgment. As experience is the best trainer for judgment, judgment-based training needs to be much more experiential in nature than the typical three ring binder and PowerPoint deck training that is so common.
Concluding Thoughts
There is a place for process and there is a place for judgment in risk management. Rarely, however, does an organization require only one or the other. This creates difficulties and paradoxes for risk management. However, it also creates opportunities for those organizations that have the confidence to let their employees use judgment when appropriate. As a general rule of thumb, simple and complicated situations call for management by processes. Conversely, complex situations call for risk management by judgment, as processes are likely to backfire or at least produce unintended and unwanted consequences.
To get the best of both worlds, the philosophy of “playbooks, not rulebooks” is perhaps the best one to adapt. It is much harder to put into a risk manual, and it may be much harder to explain and justify to board members, regulators, and outside stakeholders, but it is often the compromise that is needed for effective risk management.
There is currently a definite emphasis on trying to make risk management as objective and as rules-based as possible. In part, this is driven by a need for regulators to be seen to be doing something. In another part, it is driven by society’s false belief that in our knowledge-based economy that all problems can and need to be solved. Indeed, some risk issues can be solved, or at least effectively managed, through rules or processes, and in those situations, risk management should utilize process-based management as much as possible. However, there will always be a place for judgment as well. The fate of a drowning person may depend on the choice.
1 Atul Gawande, The Checklist Manifesto: How to Get Things Right, Picador, Reprint Edition, 2011.