Key Risk Indicator

A key risk indicator (KRI) is a metric that permits a business to monitor changes in the level of risk in order to take action. KRIs highlight pressure points and can be effective leading indicators of emerging risks. These are typically forward-looking or leading indicators.¹

Key Performance Indicator

A key performance indicator (KPI) is a metric that evaluates how a business is performing against objectives. A defined target (typically) provides the benchmark for evaluation of a KPI metric. These metrics are usually backward-looking or lagging indicators.

Key Control Indicator

A key control indicator (KCI) is a metric that evaluates the effectiveness level of a control (or set of controls) that have been implemented to reduce or mitigate a given risk exposure. A calibrated threshold or trigger (typically) brackets a KCI metric. These metrics are usually backward-looking or lagging indicators. Control indicators link with operational or process objectives.

If it is an important indicator, then it is considered key. Metrics may have multiple uses. They may inform performance, risk, or control. They are also layered for specific owners and accountable parties—building from control to process to objective, telling a story, and driving action and decisions at each discrete layer.

A Risk Taxonomy Provides Clarity

A risk taxonomy is a comprehensive, common, and stable set of risk categories that is used within an organization.

• By providing a comprehensive set of risk categories, it encourages those involved in risk identification to consider all types of risks that could affect the organization’s objectives.
• By providing a common set of risk categories, it facilitates the aggregation of risks from across the organization.

By providing a stable set of risk categories, it facilitates comparative analysis of an organization’s risks over time.

Having a risk taxonomy is critical when establishing a KRI program, which is critical for anticipating risk. It supports the relationship between metrics measuring control at a granular level with the risk they are mitigating and ultimately the relationship to strategic objectives as seen in Figure 11.1. The taxonomy also provides clarity of accountability and consistency of response and decision making within an organization across the range of stakeholders that use the KRIs.

Organizational Risk

High-level risk statements within the risk taxonomy represent how an organization might view the impact of a control failure within information security. These are essentially the things that could impact the profit and loss (P&L) by disrupting individual business processes and impacting customers. The following three are commonly viewed as risks to most organizations:

• Systems not available as expected.
• Information is exposed inappropriately (to those other than expected).
• Information is inaccurate and cannot be relied upon.

Organization leaders might not immediately care about the details of each control mitigating the various ways in which these risks could be realized; however, they do care about direct negative impact on their P&L and reputation.

Functional Risk

Cyber risk is a functional risk. This means that it is typically managed by the information security organizational function within information technology (IT). It spans the organization and requires clear integration with other functions apart from IT, such as vendor or supplier management, business continuity, and physical security, to name just a few.

Cyber risk as a functional risk type could have an impact on each of our preceding three business risks. Cybersecurity control objectives are aligned to these risks such that, if met, they would substantially reduce the risk. KRIs are designed and implemented to monitor changes in the cybersecurity risk level inherently and residually. These changes would then be reflected in the level of risk to the organization.

KRI Design Links Objectives, Risks, and Controls

KRI design begins with a clear view of the risks that the organization faces and continues with the further synthesis of these risks into control objectives and key controls (as in Figure 11.1). These elements of the risk taxonomy are manifest in the organization’s comprehensive cybersecurity program that start with policies and programs guided by industry best practices as well as applicable laws and regulations. The discrete programs form the basis for meeting control objectives which, when met, significantly mitigate the risk.

Organizational threats typically occur in the context of actors, targets, and vectors. We can illustrate how KRIs play a role in telling a story around risk, both inherently and residually, in the following way:

• Threat actor—a person or entity who impacts or has the potential to impact the security of an organization. These could be internal, external, or vendors/suppliers.
• Threat targets—the things we are trying to protect; things that are valuable to threat actors such as system working correctly, personal information, intellectual property, and so on.
• Threat vectors—paths that threat actors utilize to acquire threat targets; people (our employees, vendors, etc.) or systems or supply chain.

Table 11.1 indicates some examples of KRIs. These are aligned to high-level control objectives that are associated with threat vectors (because these are what we can control!) as well as threat actors. The KRIs are measured to provide an indication of the risk level and the strength of the program in consideration of our threat targets. In Table 11.1, KRIs that may need to be interpreted together are grouped in the column called “Examples of KRIs.” The multilayered approach to KRIs is also indicated in differentiating some examples of more detailed technical KRIs. Each of these KRI examples may also be separately categorized in one of four categories: incident counts, loss magnitude data, threat data, or control data.

Case Study Where Triggered KRIs Were Apparently Ignored

The Target data breach in 2013 affected over 110 million customers and losses upwards of $250 million.

Hackers accessed the network using an HVAC (heating, ventilating, and air conditioning) supplier’s credentials. The HVAC supplier had access to the Target network, and used it to collect temperature and energy usage data from each store. The hackers were able to get the log-on credentials using a phishing e-mail aimed at Target suppliers. They deceived one of the HVAC employees who opened the e-mail, allowing them to infect some of the supplier’s computers. The hackers then waited until the malware captured the log-on credentials for Target. The HVAC vendor did not have adequate system protection, so the breach went undetected.

The Target event illustrates the need for cybersecurity integration in programs such as vendor or supplier management and to have a KRI program that governs response to KRIs.²

Using KRIs for Improved Decision Making

The objective of a strong KRI program is to improve decision making within the organization. Ideally, this should be forward looking. Reporting and presentation of the KRIs is both art and science. Much of the art in depicting the view of information security risk is in decoupling the detailed technical metrics and tech-speak, when presenting to senior leadership and board members. Another important consideration if presenting an abundance of KRIs is to avoid this leading to a false sense of security. Context and messaging are critical.

Stakeholders Want to Be Informed

There are a range of stakeholders in an organization that interact with the metrics measured to indicate changes in risk and control levels. The key stakeholders are typically the board of directors, senior management, chief risk officer, and chief information security officer (CISO) or head of cybersecurity. Differences as to how to present KRIs are directly related to the purview of the stakeholder group. The CISO or head of cybersecurity needs to have the most complete and granular set of KRIs to effectively manage progress and continuous improvement of information security.

Board members and senior management need to understand the inherent and residual risk associated with cybersecurity, as well as the cost of control. In order to understand these metrics, there needs to be a clear relationship of the cybersecurity risk to the organization strategy and to the organization risk appetite, as this is how the inherent risk would be viewed.

Here are the key things the CEO and the board of directors want to know about:

• Cyber risk culture and awareness.
• Inherent cyber risk (i.e., before controls are taken into account). Inherent risk level is usually influenced by changes in the threat level; new threat actors, uptick in attacks and sophistication, higher value of threat targets, and so on.
• Residual risk (i.e., after controls are taken into account). The level of control we have implemented over the various threat vectors that can be indicated by cyber program status, cyber program maturity, compliance, and peer comparison.
• Actual experience and trends: incidents, losses, policy violations.
• Big-picture metrics: the KRIs that illustrate support for the strategic objectives.

Inherent Risk, Residual Risk, and Big-Picture KRIs

There are several metrics that support inherent risk evaluation. Inherent risk arises any time a tool or process can result in potential losses. The aim of security controls is to mitigate inherent risks. This category of risk includes trend of exploits and vulnerabilities across the industry, trend of losses across the industry, and trends of losses that have occurred in companies with similar business models.

Residual risk is that which remains after security controls are implemented. Yes, residual risk is evaluated through metrics for a view of the scope, maturity, and integration of the information security program internally. The KRIs that are designed to indicate the strength of the information security program will be aggregate KRIs, such as percentage of employees trained, percentage of internal infrastructure covered, trend in incidents, response time to mitigate incidents, and trends in losses.

This view of inherent risk and residual risk that is overlaid onto strategic objectives and risk appetite helps an organization understand the cost of mitigation. Table 11.1 represents a useful way to overlay and link these components.

Aligning KRIs to the big picture—that is, the strategic objectives and risk appetite—is imperative for ensuring senior management understanding and support. The way to do this effectively is to align KRIs and design reporting so that the KRIs tell the story that relates to these business objectives.

For example, consider if one of your organization’s strategic objectives is to double revenue by expanding the customer base via an online channel. Each of the risks that could impact achievement of that objective must be evaluated and mitigated appropriately. It is quite easy to see that a risk scenario of an external direct denial of service (DDoS) attack would have an impact on customer experience by impacting system availability for customer’s transactions. This impact could result in lost revenue and customer retention. From this description, a few KRIs should be apparent, including customer satisfaction and customer retention. These can further be correlated with core system uptime, and the range of control metrics associated with protecting the network perimeter.

Dashboard Samples Tailored to Stakeholders

The cybersecurity and information security disciplines measure many things. They often deploy a robust set of dashboards and reports that are targeted and focused by each stakeholder.

Figure 11.2 represents a sample high-level dashboard with some of the components listed above. These are all aggregated KRIs to provide some information about where the program is and how that relates to the actual experience. Remember that the objective of this reporting is to aid decision making, so these metrics may need to be adjusted to provide more tailored information. For example, the KRIs may be broken by business unit or geography.