Chapter 7: How Secure Are You? – Measuring Security Posture

In this chapter, we will talk about security posture. Security posture is a measurement of how ready you are to deal with a cyber-attack. The security posture is set during the development of the security strategy. Not surprisingly, the security posture is complex to measure, hard to maintain, and closely related to the value that a robust security operation brings to the business.

Traditionally, discussions about security posture have focused on reducing risk, rather than driving business value. This chapter will focus on how practitioners should have these discussions in the context of business value.

In this chapter, we'll focus on risk assessment, the security posture, the security strategy, and how security is crucial for a business. Specifically, we'll cover the following topics:

• Security as risk reduction
• Measuring risk reduction
• Strategy maps – security as business value
• Working with the security strategy map

The focus of this chapter is to discuss how risk management and strategy definition can be impacted by the agile security operations concepts that are discussed in this book.

In terms of the agile security operations loop, the approach in this chapter primarily interacts with the recovery and retrospective aspects of incident response. In other words, this chapter considers risks, reviews, and how they feed into the security posture and its strategy.

Security as risk reduction

The strategic goal of cybersecurity is two-fold: reducing risk to the business and enabling new business initiatives. We'll discuss risk reduction first.

In Chapter 1, How Security Operations Are Changing, we discussed the risk-based approach to security and how the overall residual risk to assets is based on the exposure, the level of vulnerability, and the available controls, as described in the NIST risk management framework here: https://csrc.nist.gov/publications/detail/sp/800-30/rev-1/final.

A framework for risk reduction was depicted in Figure 1.1, in Chapter 1, How Security Operations Are Changing. This figure has been reproduced here, also indicating how the various areas map to various areas of the ATT&CK framework, which we will use to develop our discussion of risk:

Figure 7.1 – Risk model from NIST 800 30r1 using the categories of the ATT&CK framework

We will discuss the management and treatment of risk in more detail shortly, through an approach based on the ATT&CK framework, which we used earlier in this book. This framework is particularly helpful in answering those questions.

The factors that play a role in this risk framework are as follows:

• Threats: Threats have threat actors who execute threat events. Threat actors are typically threat groups who run campaigns or targeted intrusions. In terms of the ATT&CK framework, threats are composed of threat actors and threat events, and threat actors are indexed in the framework.
• Exposure: Exposure can be managed by minimizing the attack surface, such as not opening unnecessary ports on the firewall, disabling any services that are not needed, or limiting the types of networks allowed to connect to a service.
• Vulnerability: Vulnerabilities exist all the time and are the result of either programming errors in our own or someone else's code, or configuration errors.
• Controls: Controls were described in Chapter 5, Defensible Architecture, in terms of roots of trust and visibility of events.
• Impact: The ATT&CK framework describes the impact of the techniques that adversaries use to disrupt availability or compromise integrity by manipulating business and operational processes (https://attack.mitre.org/tactics/TA0040/).

From this, we can determine what the key questions are for risk management in organizations.

It all comes down to a combination of impact and controls. Controls usually take the form of static defenses that block certain routes, from threat events to impact. Controls focus on threat reduction, as well as limiting exposure or a vulnerability, by severing the link between the threat actor and its impact. In terms of the five types of cyber defense, the implementation and operation of cybersecurity controls are primarily what we do as part of passive defense.

Impact determines the amount of loss an organization may suffer if the risk event occurs. In terms of this risk framework, minimizing the impact is also desirable, and this is what we aim to do in active defense, as discussed in Chapter 6, Active Defense. Defending actively is about having the resilience and response capabilities to ensure that the impact of the risk is minimized.

As an example, an attacker that has established command and control on our network and is trying to pivot to a more privileged point (an account, device, or service) on the network has already caused an impact. Their objective is to get more impact. The aim of active defense at this point is to limit the impact that the attacker has already achieved. From the viewpoint of the attacker, this is not enough. This realization is the background behind our earlier statement, in Chapter 1, How Security Operations are Changing, and Chapter 2, Incidence Response – A Key Capability in Security Operations, that defense aims to detect and contain attackers before they have reached their objective.

Cyber defense, which engages both active and passive defense, must consider risk from the following perspectives:

• The description of the risk. It is important to have an accurate understanding of what the risks that we are trying to manage are.
• A qualitative understanding of the risk by focusing on the impact that would occur if the risk materialized.
• How to translate the risk into a meaningful quantity that can be used in financial planning?
• What controls are available to manage the risk?
• Which of the available controls is the best for managing that risk?

Usually, it is not possible to fully control risk, and a certain amount of residual risk remains after all the controls have been applied. Residual risk is the risk that remains once controls and mitigations have been applied. We will understand residual risk better once we measure risk reduction.

Measuring risk reduction

In Chapter 1, How Security Operations Are Changing, we discussed a risk management framework but also noted that a risk-based approach to security runs the risk (pun intended) of overfocusing on passive defenses – that is, controls that can be implemented once and left alone.

As we stressed in Chapter 1, How Security Operations Are Changing, security operations are about more than just passive defenses; they focus on the activities that are performed during security operations that help promote resilience. In this section, we'll discuss how we need to rethink risk to take that into account.

The key to getting to an improved risk management framework is to have a reliable set of metrics that measure the resilience of an organization against cyber-attacks. This includes the measures an organization takes operationally to mitigate the impact of realized risks – that is, risks that have occurred – as well as how an organization assesses and implements risk management for future lines of business.

A robust framework for the management of risk has been developed by NIST (https://csrc.nist.gov/Projects/risk-management) and is documented in the NIST 800-30r1 document (https://csrc.nist.gov/publications/detail/sp/800-30/rev-1/final). This document is quite comprehensive and discusses the risk management framework in detail.

Description

To understand risk, it is necessary to describe it. The ATT&CK framework helps describe risks emerging from cyber events. There are several ways in which the ATT&CK framework assists in describing risk:

• Threat groups: As is apparent from Figure 7.1, risks start with threat group actors and threat group events, and the ATT&CK framework contains a list of groups and associated tactics and techniques.
• Mitigations: Another handle on risk is to consider mitigations that have not been applied to the environment, which correspond to existing unmitigated vulnerabilities.
• Impact: The third approach is to focus on impact.

  Note

  These three approaches are discussed in NIST 800-30r1 on page 15 of the document.

A useful source of information to inform risk descriptions is to focus on past attacks and assume a scenario in which one or two controls that worked in the past, have now failed.

A practical example is malware-laden emails that are being stopped at the border. Few organizations look at this in detail, but by not inspecting this type of information, they are missing out on something. The collection of emails that are being held tells a story about actors that are consistently trying to enter our environment (and up to now, have kept failing). What if, one day, they slightly improve their malware and succeed?

Understanding the cadence of these events is interesting too. As a further example, the Emotet group (largely arrested in January 2021) tended to attack in phases with different iterations of the malware (sometimes referred to as epochs). This sort of activity is easy to spot by inspecting malware queues at the border, and evidence of a large amount of malware being held should lead to increased vigilance inside the network.

Don't confuse the risks themselves with tactics, techniques, or their impact. The impact is a component of the risk framework that is primarily responsible for the financial losses associated with risk, as discussed in the next section; it is not a description of the risk itself. Suffering the impact is part of the description of the risk.

A risk heat map may be developed if we consider the combination of threat groups and mitigations, outlining where the organization is at most risk. Usually, the heat map maps the identified risks on a scale of likelihood/impact.

Financial aspects of risks

Using heat maps is one way to model risk, but a heatmap can be hard to contextualize into a business context. Having a list of risks labeled red, amber, or white, along with some risk descriptions, does not give an organization good guidance on how to treat these risks and gives only a hazy understanding of the potential impact of these risks on the financial bottom- line.

To understand the meaning of a risk to the business, it needs to be understood in financial terms. To understand the financial impact of a risk, it is necessary to understand its business impact and technical impact first.

Technical impact starts with systems and can be related to the impact column in the ATT&CK framework. This may be loss of data, exfiltration of confidential or private information, destruction of systems (and equipment in extreme cases involving control systems), or hijacking resources, leaving the cost of their use to the business.

The business impact is described in terms of the direct costs that occur when a system is hijacked, or the costs incurred when a process stops working, the reputation lost, or the revenue that's not earned if a system is unavailable.

Measurement in Cybersecurity

A useful set of recipes and approaches to translating the traditional risk approach into business terms can be found in - How To Measure Anything in Cybersecurity Risk, Douglas W. Hubbard and Richard Seiersen, Wiley, 2016. The tools discussed in this book, as well as more information on this approach, are available on the associated website: https://www.howtomeasureanything.com/cybersecurity/.

To approach risk from the business impact end, a sound understanding of process and systems architecture, as well as a threat model, is a good starting point. This type of information enables an organization to perform an impact analysis, which outlines, in business and process terms, what would fail if a certain risk occurred.

How to understand and describe these models was discussed in Chapter 5, Defensible Architecture.

Controls

The next two subsections focus on controls – both the controls that are available to manage risk and selecting the best controls to manage the risk. Cybersecurity controls that may be used in risk management are cataloged in the NIST 800-53 series. This is a base catalog of controls called a control baseline. From this baseline, organizations may select a subset of controls that are relevant to them. This is called an overlay.

The 800-53 control baseline is best viewed as a collection of possible controls, but not something that needs to be implemented in its entirety. Specifically, it will need modification, interpretation, and specific operationalization to fit in with the environment that it will be applied to.

The NIST control set works with a collection of overlays that assist organizations in defining the control set based on the scenario they face. As an example, the industrial control system (ICS) overlay provides a recommendation for the controls to implement in this scenario. The things that an overlay will specify are, usually, the following:

• Controls that need to be deleted, added to, or modified from the baseline.
• Technology-dependent applicability and interpretations of controls.
• Specification of certain acceptable values for the control.

In this way, control overlays can manage which controls are implemented and how they are implemented, and they can contain any amount of specific technical detail referring to a system, even down to specific acceptable values for certain controls.

NIST maintains a comprehensive website on risk management (https://csrc.nist.gov/projects/risk-management) that focuses on control selection for certain scenarios.

Controls and policies may look like quite static, non-agile ways of running security operations. Nevertheless, they are essential for communication with the business, as well as managing the passive elements of security.

Control frameworks and risk management may also drive aspects of a more active mode of defense, where defenders specifically seek out and interrogate violations of policies, un-monitored controls, or control violations. This is often useful for several reasons:

• It assists security operations significantly with gaining and understanding the context in which the business operates, as well as outlining some future rabbit holes that attackers may attempt to slip through. All in all, this sort of work helps with understanding the map of possible attack paths through the business.
• Highlighting and measuring gaps in controls and modifying the control overlay in use by the business.
• Deriving additional controls and monitoring points based on the analysis of the attack path.

In addition, it is also useful to consider the ATT&CK mitigations in the context of a control framework to determine the sort of attacks that the adopted control framework is capable of preventing, monitoring and resolving.

Risk management versus enabling the business

There is more to risk than just reducing already existing risks. The security author Derek Brink calls these already existing risks. Unrewarded risks are risks that exist in the business as it is today and taking these risks does not add to new revenue or improvements in revenue.

You can also think about risk management in terms of rewarded risks – calculated risk-taking to support the new opportunities that can be created once security is done right.

This leads us to a brief excursion on security strategy, which is the subject of the next section.

Strategy maps – security as business value

It is a somewhat stale statement that security should contribute to the business to enable new business initiatives. You might also say this by stating that security should be an enabler of the business, rather than a blocker. Yet this is easier said than done – it is hard to determine how security enables the business.

To map out how an activity contributes to the wider goal of an organization, especially in cases where conflicting activities and goals need to be balanced, a compromise created, or a new innovative solution found, businesses need to develop a strategy.

Strategy is a commonly misunderstood term. A strategy is not a plan. A strategy is what you need when you're dealing with a situation in which outcomes can be uncertain based on the actions of others. A good strategy considers scenarios and is grounded in a deep understanding of the drivers of business processes, value chains, and how attacks compromise these chains.

The History of Strategy

As Lawrence Freedman argues in his Strategy, A History (Oxford University Press, 2014), strategy only became separated from tactics once the size of the armies necessary to wage war had grown to such a size that a separate staff department was required to manage the logistics and planning for an army of that size. The term then made it from warfare to business.

Strategy is about how an organization aims to reach its goals in an environment where the actions of other players in that environment are uncertain, multiple future scenarios might be possible, and the organization has objectives that are, at times, in conflict. It is because of this that strategy is not a plan. Not everything is under control.

This somewhat uncertain nature of strategy development leads to a widespread mystification of what the strategy process entails, which leads to many organizations getting the strategy wrong. The common pitfalls that hamper mature strategy discussions in many organizations are as follows:

• Having strategy discussions without the corresponding visibility of the environment
• Not modeling risk properly
• Not understanding the business context properly

All in all, risk management, as discussed in the previous section, is key to the development of a sound security strategy. As we have seen, to perform risk calculations properly, a significant amount of visibility is required to assess the operation of a control framework, and the approach of assuming scenarios in which current controls fail is a useful way to assess the impact of cyber threats to the business based on real attack data.

Risk modeling and business context are the two key areas where insights gleaned from security operations have considerable strategic impact. To avoid the common pitfalls of strategy development, it is necessary to have visibility into the environment and a risk framework.

Strategy Maps

To capture how strategy contributes to organizations, in 2001, Robert Kaplan and David Norton expanded their model of the balanced scorecard to strategy maps in their book The Strategy-Focused Organization (The Strategy-Focused Organization: How Balanced Scorecard Companies Thrive in the New Business Environment, Boston, MA: Harvard Business School Press, 2000).

The tool that we will use to discuss matters of security strategy is the strategy map, which is an evolution of the balanced scorecard.

Constructing strategy maps

As we discussed in the previous section, creating, and setting up a strategy is not a simple matter. Strategy is not a plan, but rather the definition of a specific focus or set of responses to imaginable scenarios.

As an example, the four goals of incident response, as discussed in Chapter 1, How Security Operations Are Changing, form a strategy for dealing with incidents, so, they are a good starting point for agile security operations.

The complexity of strategy is not the entire story, however. Strategy focuses on three specific aspects of the business, which roughly translate to the why, the what, and the how of the business, which are as follows:

• Value creation: Why is the business there at all?
• Business processes: What processes underpin the process of value creation?
• Capabilities: How are these processes operated and are there specific gaps that need to be filled?

In addition, a strategy also considers how all these elements are aligned.

Strategy map layers

The strategy map has several layers, indicating that it looks at the totality of a business from multiple points of view. These layers are as follows:

• Financial or value
• Customer
• Operations
• Capabilities or learning and growth

In the strategy map, the discussion of value creation, processes, and capabilities is focused on these four layers: financial, customer (both representing value creation), operations (representing the business process focus), and capabilities (representing the how).

For a cybersecurity strategy, value, processes, and capability are similarly relevant. The four goals of incident response form the why of security operations. The what is covered in the five types of cyber defense and the processes that make up security operations, while the how is covered in the capabilities that deliver it all. The latter will be discussed in more detail in Chapter 9, Running and Operating Security Services.

Security strategy maps

Is a strategy map for a security strategy possible? The answer to this question relies on whether we can capture the value of a security program in terms of financial, customer, operations, and capabilities metrics. We are generally not used to discussing the value of a security program in financial terms or customer perspectives. As security practitioners, we are generally pretty good at discussing the operations or capabilities of a program. Hence, developing a strategy that works from a business perspective has traditionally been somewhat of a challenge.

Strategy Maps for Security

A series of articles by Derek Brink on the securityintelligence.com website contains a development path for a security-based strategy map that follows the Kaplan and Norton model. See https://securityintelligence.com/a-strategy-map-for-security-leaders-applying-the-balanced-scorecard-framework-to-information-security/ for the start of the series. This forms a good starting point to start thinking about a strategy for security.

In terms of agile security operations, a strategy may be articulated as a set of agile principles, practices, and benefits that are typical of agility in security operations, in addition to the strategy that already exists.

At a high level, we may capture the development of a security strategy with the strategy map in the following terms:

• The financial or value of agile security operations should be expressed in terms of the improved defensibility of the business, expressed in terms of either risk buydown or new initiatives that can be enabled once security is in place.
• The customer perspective can be captured in terms of more rapid recovery after an incident, or a reduction in the need to perform incident response at all, since incidents are caught earlier and with better fidelity.
• The operations perspective can be augmented with the set of practices we discussed in Chapter 6, Active Defense.
• Capabilities or learning and growth focus on how we learn from past incidents, as well as the specific practices and incidents gained with agile security operations.

This is a good starting point for organizations that already have a security strategy and are looking for specific points of view emerging from agile security operation to augment how they articulate and communicate strategy.

But what about organizations that must start from scratch and have nothing at all? The following section contains a few useful pointers.

Starting a security strategy

For organizations at the beginning of the security journey, it can be hard to determine how to start. This section outlines some high-level steps that can be used to start work on a security strategy:

1. Get some visibility of the current environment and the elements that are currently in place.

   Note

   Most organizations have firewalls, for instance. Are the firewall logs available to get some more information on what is currently happening? What is currently allowed or blocked by the firewalls that looks like malicious activity?

2. Use this information to understand and map the threats to business processes using the scenario planning method mentioned previously. Focus on what happens if a control or measure fails.
3. Translate those threats into risks and prioritize the risks.
4. Understand and map the necessary controls to counter those threats using, for instance, ATT&CK.
5. Develop a plan of action to implement those controls.

This is a bootstrapping method of the security strategy that uses already existing information, together with some imagination and the work that's already been done in developing the ATT&CK framework, to develop an improvement plan.

With these improvements, the visibility into the environment will also increase, leading to a virtuous cycle of improvement in the security posture.

Now that we have defined the outline of a strategy map, we'll consider how to work with them, and what they mean in practice for a security program.

Working with the security strategy map

Another way to capture the benefits of a strategy is to have a value map. This maps out how valuable an initiative – in our case, agile security operations – is to an organization, as well as how effective the proposed principles are.

In this section, I will outline a few metrics that organizations can use to map the effectiveness of their strategy onto a set of metrics in terms of financial measures, customer measures, operations, and capabilities. The latter focuses on the effectiveness of security operations and indicates the means of improvement.

Financial metrics

The financial metrics of the security strategy are primarily focused on measurable risk reduction, which was discussed earlier in this chapter. Among the specific financial metrics to measure the quality of a security program, you can consider the following:

• Risk reduction measured in financial terms, such as incidents avoided
• New revenue made possible by improved security practices
• Improvements in the business alignment of security and business processes

A practical way of getting a handle on some of this data is to use scenario planning. To start this process, consider the past incidents that the organization has experienced alongside their frequency and actual impact. Then, imagine that, during these incidents, one or more of the controls had failed. Now, consider the impact the same incident would have had under that scenario.

It is also possible, and useful, to expand this scenario planning into a tabletop exercise that runs through a scenario on paper and aims to determine what the organizational response to events would be. This gives a good handle on missing elements in the incident response plan but can also serve to socialize the potential consequences of a cyber incident.

Note

As an example of this, consider malware that's been stopped by the malware software. What would the consequence be if the malware had been able to execute? What would the next control be under that scenario?

Such scenarios can give us a handle on both the potential impact of incidents, as well as the cost-effectiveness of the various controls that are being used in the business.

We'll look at customer metrics next.

Customer metrics

As a set of customer metrics, we may consider the following:

• Cyber-attacks that are avoided by controls. For instance, filtering at the border stops many email-borne spam and malware attacks that would cost time and effort to resolve if they made it into the environment.
• Vulnerabilities affecting customer environments and the speed with which they are addressed.
• The quality and quantity of cyber advice to users in the environment.

Customers are somewhat hard to define in the context of a cybersecurity program, but in organizations that do not sell software products or IT services, they generally consist of end users in the environments we defend.

Operations metrics

Most organizations measure cybersecurity primarily in terms of operations metrics: vulnerabilities patched, the amount of malware removed by the detection and response software, or the number of bad emails held in the queues at the border.

Note

Carson Zimmerman defined several robust metrics to measure the effectiveness of security operations (https://www.fireeye.com/content/dam/fireeye-www/summit/cds-2019/presentations/cds19-executive-s03b-practical-soc-metrics.pdf). Not all these metrics are what you would expect, and it is an illuminating exercise to consider whether you can implement these controls in your environment and how useful that might be.

We can subdivide these operations metrics into control effectiveness and detection effectiveness.

Measuring control effectiveness

The control framework is the series of controls, alongside the frequency with which they are monitored. As an example, an organization may opt to review failed logons every day and report on the top five accounts that generate the most failed logins. Such an activity is called a control, and the documentation of all controls is known as the control framework.

Control effectiveness measures several aspects of the control framework, such as the following:

• Coverage: The ATT&CK framework gives us a clue about how well all our controls cover the known and documented TTPs of attack groups. In addition, coverage can be measured as a percentage of the systems that fall under the control framework.
• Feed health: Feed health monitors the current state of the data feed that underlies the generation of alerts. Feed health considers, for instance, whether the feed is still up and running, as well as whether the format of the feed can still be parsed by the software that generates the alerts.
• Up-to-dateness: When was the control framework last reviewed?

Control effectiveness is not regularly monitored by many teams but is an important factor in determining the effectiveness of security operations. Logging deficiencies and gaps in the controls that are being monitored by a security team are major contributors to difficulties during forensics.

Measuring detection and response effectiveness

Some measurements for detection and response might be as follows:

• Incident resolution across each phase of the incident response cycle. How long does it take to detect an incident? What detected the incident? How long did it take to analyze? Did the analysis lead to another detection? How long does an incident take to resolve?
• The speed of incident resolution is indexed by the frequency of the incident. Incidents that occur frequently should be resolved fast. The cadence of response should be faster for incidents that occur more frequently.
• Detection effectiveness. With a mature detection engineering practice, it is also possible to develop a metric of how often certain detections produce an alert and whether these were false or true positives.

This discussion of control effectiveness and detection and response effectiveness completes our section on operations metrics. This also reinforces the fact that what matters in security operations is detection and response capabilities.

Metrics for capabilities

The capabilities dimension measures the personnel of the security operation. Generally, capabilities are measured through the maturity model of an organization. For security operations, several freely available maturity models are available.

Maturity Models for Security Operations

There are several maturity models for security operations, among which is the SOC CMM by Rob van Os (https://soc-cmm.com/) and the Open CSIRT SIM3 model (https://opencsirt.org/csirt-maturity/sim3-and-references/).

Some metrics that may be useful, as well as easy to measure and record, are as follows:

• Training
• Experience
• Trust group membership

These are just some examples of metrics that may be useful in measuring the effectiveness and value of agile security operations. There is no doubt that more could be measured, and that context determines, in large, what is useful to measure.

Summary

In this chapter, we looked at several elements that make up the risk management and strategy components of security operations. We discussed how, for some of these, we can bring in an element of agility to connect to the agility needed in security operations.

This chapter also concludes Part 2 of this book, which focused on the basic aspects of implementing agile security operations.

Much of what we discussed in Part 2 forms the foundation for Part 3. In the next chapter, we will focus on some specific aspects of security operations, such as purple teaming, threat intelligence, and running security operations as a set of services.