Chapter 6. Using CERT-RMM

There are many effective and appropriate ways for an organization to use CERT-RMM to guide, inform, or otherwise support improvements to its operational resilience management activities. For those familiar with process improvement, CERT-RMM can be used as the body of knowledge that supports model-based process improvement activities for operational resilience management processes. However, not all organizations embrace the term process improvement and instead are simply looking for a way to evaluate their performance or organize their practices. All of these uses of CERT-RMM are legitimate.

In this chapter, we briefly explore the ways in which an organization could use CERT-RMM and provide a broader understanding of the concepts that help an organization determine how to make use of CERT-RMM to meet its unique needs. Section 6.1 provides selected examples of how the model can be effectively used by an organization. One such example is to use CERT-RMM to support model-based process improvement, a process that is more fully described in Section 6.2. Section 6.3 details a number of decisions around the scope of a CERT-RMM-based improvement effort, such as the organizational scope (which business units are involved), the model scope (which process areas are included), and the capability level targets (selecting “performed,” “managed,” or “defined” as the target for each process area). Using the model as a basis for diagnosis can be accomplished in a variety of ways, ranging from a formal appraisal to an informal review, as described in Section 6.4. Gaps that may be revealed through diagnostic methods should be analyzed in consideration of the improvement objectives to make sure that closing the gaps would be of value to the organization. Part of planning improvements to existing practices or planning the implementation of new practices is to determine where in the organization the practices will be performed or instantiated. Gap analysis and implementation planning are discussed in Section 6.5.

6.1 Examples of CERT-RMM Uses

This section provides several examples of how CERT-RMM can be used. This is not a complete list, but it provides insight into how CERT-RMM can be applied to a broad set of challenges and objectives. The examples given describe using CERT-RMM to

• support the achievement of strategic and operational objectives

• evaluate, guide, and compare the implementation of resilience activities

• organize and structure the use of many codes of practice

• catalyze model-based process improvement

6.1.1 Supporting Strategic and Operational Objectives

CERT-RMM can be used as a source of guidance and information to support the achievement of specific objectives related to security, business continuity, IT operations, or managing operational risk in general. Organizational objectives that are directly or indirectly tied to resilience management activities can be strong drivers for CERT-RMM-based improvements.

Such objectives may be high-level and strategic. For example, consider an organization that sells various products both online and in its brick-and-mortar stores. The organization has established a strategic objective to increase the relative percentage of online sales by 25% over three years. Operational risk has been identified as a key constraint to the strategy—publicity associated with security breaches and downtime associated with business continuity failures could severely impede the achievement of the strategic objective. This organization can use CERT-RMM to guide the convergence and improvement of its security and business continuity processes to control and manage operational risks that could undermine achievement of this strategic objective.

Such objectives could also be more tactical. For example, consider an organization that recently suffered financial losses when information systems were offline following a security incident. Prior to the incident, warning signs were clear but had not been recognized. During the incident, confusion and ad hoc procedures resulted in longer downtime. The organization now understands that both its monitoring activities and its incident management activities have to be improved to avoid such losses in the future. CERT-RMM can be used to determine the degree of improvement necessary, guide these improvements, and measure the extent to which the improvements are institutionalized.

6.1.2 A Basis for Evaluation, Guidance, and Comparison

CERT-RMM is the codification of an extensive body of knowledge. It includes

• security practices and security management experience from CERT and other reputable organizations and thought leaders that have been developed based on years of work with public- and private-sector organizations on security improvement

• business continuity and disaster recovery expertise from numerous financial industry organizations whose survival is critically dependent on the maturity of these capabilities

• converged security, business continuity, and IT operations practices from numerous practice bodies and standards

Many professionals with responsibilities for their organization’s operational resilience activities will find the model to be a useful basis to support the design, review, and comparison of such activities. Such guidance can be particularly useful when converging existing practices or when implementing new activities.

For example, consider an organization that has recently experienced an increase in access problems: employees with appropriate credentials have been unable to access certain systems and facilities. The team that has been assembled to diagnose the problem and propose improvements can use the Access Management (AM) and Identity Management (ID) process areas as reference sources for evaluating the current practices. If deficiencies are discovered, the model can be used as a source of guidance for improving practices or implementing new practices to address the issue.

Organizations and groups will also find the model to be a useful basis for characterizing, comparing, and learning from one another’s practices. Diagnostic activities as described in Section 6.4.1 can be used as a basis for formal or informal comparisons among organizations of their respective implementation and institutionalization of resilience activities. Formal benchmarking can be a valuable activity for industry groups to evaluate their collective resilience posture or for the components of a large enterprise to ensure that the overall enterprise is similarly prepared. Informal comparisons can also provide insights and support information sharing among a group of organizations.

6.1.3 An Organizing Structure for Deployed Practices

Many organizations have implemented practices from best-practice bodies or standards related to security, business continuity, and IT operations. Sometimes, such organizations discover that these practices

• might not be providing the benefit that the organization expected

• may be performed less consistently than when first implemented

• might have eroded in their effectiveness because the organization has changed or the operational risk environment for the organization has changed

CERT-RMM can be used to guide the implementation of a process superstructure that will serve to refresh, institutionalize, integrate, streamline, and give purpose to the practices that have already been implemented. The concept of a “superstructure” is not meant to imply an additional layer of activities, though that might be appropriate in some organizations and in some circumstances. An effective and efficient process superstructure can be implemented by following the guidance in the model for converging the various operational risk management practices to ensure that they are based on common and consistent risk assumptions and that they are being performed to support organizational objectives.

The model can also be used to support the institutionalization of existing practices to ensure that they are reliably and consistently performed, especially during times of stress, and without dependence on specific people or operating parameters that may not be present during a time of stress.

6.1.4 Model-Based Process Improvement

By far, organizations will find CERT-RMM most beneficial for process improvement. The unique aspects of CERT-RMM—the process focus and the capability dimension—were developed to help organizations evolve to a more enlightened treatment of managing operational resilience and sustaining capabilities over the long run. Regardless of the scope of improvement—a single aspect of operational resilience such as incident management or a comprehensive and broad view that incorporates all 26 process areas—CERT-RMM was built to enable an organization to easily begin a process improvement approach.

6.2 Focusing CERT-RMM on Model-Based Process Improvement

Most process improvement efforts can be structured to answer some variation of the following four questions:

• How do I decide what to do and in what order?

• How do I do it?

• How do I know if what I did worked?

• How do I decide what to do next?

These four questions can be directly mapped to the Plan, Do, Check, Act (PDCA) cycle, which was based on W. Edwards Deming’s Shewhart cycle [Deming 2000, Imai 1986]. Effective methods for improvement and management of change typically use some variation of this approach. This section starts with identifying the impetus or stimulus for change and making the business case to initiate a process improvement program. It then describes an effective process for initiating any organizational change. Specific considerations for CERT-RMM–based process improvement are described in Section 6.3.

6.2.1 Making the Business Case

In today’s business climate, organizations are constantly dealing with the demand to do more with less. The resources required to run the business, let alone to invest in new initiatives, are always at a premium—time, money, staff expertise, information, technology, and facilities, not to mention energy and attention span. All investment decisions are about doing what is best for the organization (and its stakeholders). However, what is best is sometimes hard to define, hard to quantify, and even harder to defend when the demand for investment dollars exceeds the supply.

Business leaders are increasingly aware of the need to invest in operational resilience—to better prepare for and recover from disruptive events, to protect and sustain high-value services and supporting assets (information, technology, facilities, and people) that are essential to meet business objectives, and to satisfy compliance requirements. So how do we ensure that investments in operational resilience will increase our confidence that services will continue to meet their mission, even during times of stress and disruption? And by so doing, how are we able to justify such investments to senior managers?

Making the business case for operational resilience, and specifically for investing in the adoption of CERT-RMM processes, is accomplished by articulating the business need and showing how CERT-RMM meets it—in a tangible and measurable way over a reasonable period of time for an affordable cost with a positive return. A well-articulated business need is the driver and stimulus for change. In the context of operational risk, it is often the answer to the question, Where does it hurt the most, or what high-impact, high-loss event(s) would put us out of business? A key step in this process is to identify the senior manager who most cares about the answers to these questions and to make sure he or she is on board as the visible champion and sponsor of the CERT-RMM improvement program.

In addition, those making the case for operational resilience must be able to demonstrate that investments are subject to the same decision criteria as other business investments, so that they can be prioritized, evaluated, and traded off in a similar fashion. Again, this ties back to business mission, strategic objectives, and critical success factors, which are the basis for determining the high-value services that support the accomplishment of strategic objectives (refer to the Enterprise Focus process area). Protecting and sustaining high-value services is the name of the game.

Once the business need is agreed to and a decision is made to take action to meet it, what is needed next is a process for ensuring that the need is met.

6.2.2 A Process Improvement Process

In large part, process improvement is about managing change, whether intentional or unintentional (including change caused by a disruptive event). The SEI has adapted Deming’s PDCA approach into a method for technology adoption and software process improvement called IDEAL. The IDEAL model is an organizational improvement model that serves as a roadmap for initiating, planning, and implementing improvement actions [McFeeley 1996]. It is named for the five phases it describes: initiating, diagnosing, establishing, acting, and learning, as shown in Figure 6.1.

Figure 6.1. The IDEAL Model for Process Improvement

The catalyst that causes an organization to execute IDEAL is described above: identifying a business need, making the case for meeting it, and using it as the impetus or stimulus for change. This can include an objective to be met (such as those noted in Section 6.1.1), unanticipated events or circumstances, a new compliance requirement, or a problem to be solved, such as a poor organizational response to a disruptive event or a security breach.

Critical groundwork is completed during the initiating phase. The business reasons for undertaking the improvement effort are clearly articulated. The effort’s contributions to business goals and objectives are identified. The support of managers who will serve as visible sponsors and champions for the effort is secured, and resources are allocated on an order-of-magnitude basis. Finally, an infrastructure for managing implementation details is put in place.

The diagnosing phase builds upon the initiating phase to develop a more complete understanding of the improvement effort. During the diagnosing phase, two characterizations of the organization are developed: the current state of the organization and the desired future state. These organizational states are used to develop an approach for improving business practice. The CERT-RMM capability appraisal is focused on the diagnosing phase of IDEAL. (See Section 6.4.1 for more details on the appraisal process.)

The purpose of the establishing phase is to develop a detailed work plan. Priorities are set that reflect the recommendations made during the diagnosing phase as well as the organization’s broader operations and the constraints of its operating environment. Specific actions, milestones, deliverables, and responsibilities are incorporated into an action plan.

The activities of the acting phase help an organization implement the work that has been conceptualized and planned in the previous three phases. These activities will typically consume more calendar time and more resources than all of the other phases combined.

The learning phase completes the improvement cycle. One of the goals of the IDEAL model is to continuously improve the ability to implement change. In the learning phase, the entire IDEAL experience is reviewed to determine what was accomplished, whether the effort accomplished the intended goals, and how the organization can implement change more effectively and/or efficiently in the future. Records are kept throughout the IDEAL cycle with this phase in mind. These include CERT-RMM work products such as changes to resilience requirements, updates to service continuity plans, and incident reports.

As with any process improvement activity, some phases and activities such as those described for IDEAL are generic—they can be interpreted and applied with minimal customization based on the specific improvement initiative. Correspondingly, there are phases and activities that will require interpretation and tailoring when considering CERT-RMM in its entirety or when the focus of improvement is on specific process areas such as Incident Management and Control or Service Continuity. The remainder of this chapter describes some unique considerations or applications of the IDEAL model when using it as the basis for improving operational resilience management processes as defined in CERT-RMM.

6.3 Setting and Communicating Objectives Using CERT-RMM

A key element of any improvement effort is to establish and communicate clear improvement objectives. In addition to the stimulus for change or business objectives for change described in Section 6.1.1, objectives for a CERT-RMM–based improvement effort should include a clear delineation of scope. Scoping an improvement effort includes two key parts: the organizational scope and the model scope. The organizational scope is simply the part of the organization or an activity of the organization that is the focus of the improvement effort. Section 6.3.1 describes the elements and terminology of organizational scoping. The model scope is the designation of which parts of CERT-RMM will be used to guide the improvement effort. Section 6.3.2 provides information about how to establish a model scope and describes both coarse-grained and fine-grained scoping options that are available in CERT-RMM.

Most improvement efforts will include capability level targets for selected CERT-RMM process areas. Establishing such targets is an effective and efficient way to communicate the extent of process institutionalization that is desired for the organization. Section 6.3.3 provides information about establishing and communicating capability level targets.

When scoping an improvement effort or establishing capability level targets for an improvement effort, it is important to consider the following:

• Organizational or strategic objectives— Both the organizational scope and the model scope should be set in the context of the organizational or strategic objectives that are driving the change. Some parts of the organization might be more or less appropriate for inclusion in the scope based on such objectives. The parts of the model that are included in the model scope should be closely aligned to the overall objectives. Remember that the organizational or strategic objectives can be diverse—they can be as simple as improving sales or as complex as preventing further data breaches or denials of service.

• Timing— The scoping and objectives for an improvement effort may change over the course of time as a result of planned or unplanned changes to the organization or its operating environment. It may also be appropriate to establish a time-phased approach for both scope and objectives to ensure that the improvement effort is able to generate visible results quickly enough to be sustained (in other words, consider tackling low-hanging fruit to generate some quick wins to build momentum and support).

• Regulatory mandates or industry initiatives— Sometimes the driver for change comes from outside the organization in the form of a new regulatory mandate or industry initiative. In these cases, both the organizational scope and the model scope may be determined by the external driver. A phased approach that expands the organization and model scope over time may be appropriate to ensure that the approach for dealing with an external driver is consistent with and supports business objectives (versus being a compliance checklist exercise).

• Sponsorship— Scoping should always be established with a careful consideration of sponsorship. The organizational scope should generally be aligned to the organizational reach or influence of the sponsor, and the model scope should generally be aligned to the responsibilities of the sponsor. It may also be appropriate to consider a phased approach to sponsorship in which successive layers of sponsorship are identified and secured as the scope of the effort increases over time.

For any improvement effort or CERT-RMM deployment, it may be appropriate or necessary to iterate the selection of organizational scope, model scope, and capability level targets in order to optimize them to the objectives and sponsorship for the improvement.

6.3.1 Organizational Scope

The organizational scope is the part of the organization that is the focus of the CERT-RMM deployment. In broad terms, the organizational scope should be bounded so that there are clear lines drawn for what is included in the improvement activities. This section presents some language and conventions that can be used to establish and describe the organizational scope.

The simplest scheme for organizational scoping is to focus on an explicit part of the organization. However, an organization may choose to bound the improvement effort around a specific system (such as the payroll system), a network, or a specific service, or according to another convention that is consistent with the improvement objectives. For example, an organization that had a data breach of a classified system might bound the improvement effort around that system. Thus, the effort would focus on the services provided by the system (which must meet their mission consistently) and the assets related to the system. The effort might also include the organizational units that have responsibility for managing the system and ensuring its resilience.

CERT-RMM has a strong enterprise undertone. This is because effective operational resilience management requires capabilities that often have enterprise-wide significance, such as risk management. However, the enterprise nature of CERT-RMM should not be interpreted to mean that it must be adopted or applied at an enterprise level. On the contrary, CERT-RMM can be most effective when applied to a well-defined organizational scope and where enterprise influences can be measured.

The following terms can be used to describe the organizational scope and will be used in Section 6.5 to describe planning issues associated with CERT-RMM deployment:

• Organizational unit—a distinct subset of an organization or enterprise. Typically, the organizational unit is a segment or layer of the organizational structure that may be clearly designated by drawing a box around part of the organization chart.

• Organizational subunit—any sub-element of an organizational unit. An organizational subunit is fully contained within the organizational unit.

• Organizational superunit—any part of an organization that is at a higher level than the organizational unit.

The organizational scope is established by clearly identifying one or more organizational units that will be the focus of the improvement.

Figure 6.2 shows the typical relationships among organizational unit, organizational subunit, and organizational superunit on a generic organization chart. In this example, the organizational unit is defined as a specific segment of the organization as shown on the organization chart with multiple subunits. In this example, the term organizational superunit can be used to refer to element 1 on the organization chart, as shown; it can also be used to refer to the entire organization.

Figure 6.2. Organizational Unit, Subunit, and Superunit on an Organization Chart

For some improvement objectives, it may be optimal to designate an organizational unit that comprises all of the parts of the organization that are directly involved in the delivery of a specific service or that are responsible for a specific system. On an organization chart, such an organizational unit would be indicated by selecting the various elements of the organization that are responsible for the service, as shown in Figure 6.3. In this case, the term organizational subunit is less meaningful but could still be used to refer to elements such as 1.1.2 or 1.3.3.1. The term organizational superunit can be used to refer to element 1 or to the entire organization.

Figure 6.3. Alternate Organizational Unit Designation on an Organization Chart

6.3.2 Model Scope

The model scope represents the parts of CERT-RMM that will be used to guide the improvement effort. In other words, the model scope specifies which parts of the model will be deployed in the organizational units that compose the organizational scope.

The model scope is determined by selecting specific CERT-RMM process areas. Process areas should be chosen based on the objectives and business case for the improvement effort and in consideration of the other factors described above such as timing, regulatory mandates, and sponsorship.

For example, the organization described in the first example in Section 6.1.1 might choose the following process areas as its initial model scope to help manage operational risk in support of its online sales growth objective:

• Service Continuity (SC)—to ensure that business continuity practices are adequate to sustain the operation of its online sales infrastructure

• Knowledge and Information Management (KIM)—to improve the protection of customer information

• Risk Management (RISK)—to establish common guidelines for risk tolerance and procedures to evaluate and mitigate identified risks in a consistent manner

• Communications (COMM)—to institute procedures and guidelines for communications that will support the organization’s objective to preserve customer confidence even during times of stress

Similarly, the organization described in Section 6.1.1 as having suffered financial losses due to a security incident might choose the following process areas as its model scope to facilitate improvements to its incident management process and to implement more effective monitoring capabilities:

• Incident Management and Control (IMC)—to ensure that appropriate practices are institutionalized to support incident response

• Monitoring (MON)—to consistently instrument and monitor its operational environment so that potential threats can be identified early

Both example organizations might choose additional process areas in later phases of an improvement effort or might identify additional needs resulting from implementing improvements in these initial process areas.

There are no firm rules about the minimum or maximum number of process areas that should be selected to include in the model scope. Care should be taken to select as many process areas as needed to achieve the objectives, but few enough that progress can be demonstrated in a reasonable time frame for the sponsor and key stakeholders. If the objectives require a large number of process areas, then a time-phased approach should be considered.

6.3.2.1 Targeted Improvement Roadmaps

Targeted improvement roadmap (TIR) is a term that is used to designate a specific collection of CERT-RMM process areas that serve a particular improvement objective. An organization could declare a TIR to represent its unique objectives for managing operational resilience, or it might use a TIR that was designed by another organization or group. Industry groups might establish TIRs to represent their specific operational resilience concerns or to address an industry initiative or new regulatory mandate. Also, an organization could establish TIRs for specific tiers of suppliers or external dependencies and use the TIRs to support the evaluation, selection, and monitoring of those entities. Appendix B contains several example TIRs.

In some cases, it may be appropriate to establish a finer-grained model scope than can be set by choosing entire process areas. CERT-RMM provides for several fine-grained scoping options that can be used in such cases, as described below.

6.3.2.2 Practice-Level Scope

Practice-level scope enables the model scope to be limited to selected specific and generic practices within a process area. This option does not have to be applied to all process areas when establishing the model scope, but it may be appropriate for one or more process areas to address specific improvement needs or concerns. This scoping option may be useful in the early phases of an improvement effort, in response to very narrow improvement objectives, or to be consistent with the span of influence of the improvement sponsor.

For example, suppose that an organization’s improvement objective is focused narrowly on information technology disaster recovery activities. From the Knowledge and Information Management (KIM) process area, the organization might choose to include only specific practices KIM:SG5.SP3, Verify Validity of Information, and KIM:SG6.SP1, Perform Information Duplication and Retention, because it is concerned about its information backup practices and about ensuring the validity of information assets that will be used during disaster recovery operations.

6.3.2.3 Asset Scope

Because CERT-RMM addresses four asset types—people, information, technology, and facilities—the scope of the improvement effort could be focused on one or more process areas that could be tailored to focus on one or more asset types. For example, if the Asset Definition and Management process area is chosen, the scope of application of this process area could be limited to the “information” asset. Some process areas are already bound by an asset scope. These include Human Resource Management and People Management (people), Knowledge and Information Management (information), Technology Management (software, systems, and hardware), and Environmental Control (facilities). This option may be useful based on certain improvement objectives, a phased improvement strategy, or to tailor the model scope to best fit the span of influence of the improvement sponsor.

For example, an organization may limit the asset scope for phase 1 of a multi-phased improvement project to information and technology assets only. This is consistent with the span of influence of the improvement sponsor and with the immediate organizational objective related to improving information security. If the model scope for the improvement project includes the Asset Definition and Management (ADM) process area, for phase 1 of the effort, ADM will be applied to information and technology assets only.

6.3.2.4 Resilience Scope

CERT-RMM addresses the convergence of three broad categories of operational resilience management activities: security, business continuity, and IT operations. Resilience scope is an option that limits one or more process areas to a subset of these resilience activities. This scoping option is useful in organizations where convergence of these activities is not yet occurring or where convergence is an organizational objective.

For example, an organization in which business continuity, security, and IT operations activities are still compartmentalized may initiate an improvement effort that is sponsored by the information security manager. The organization can use the resilience scope option to limit the interpretation of selected process areas so that they apply to security activities only. If the model scope includes the Compliance (COMP) process area, for example, it would be interpreted to apply exclusively to security-related compliance obligations.

Figure 6.4 shows the relationship of the four model scope options.

Figure 6.4. Model Scope Options

6.3.3 Capability Level Targets

Capability levels are used in the model to describe the achievement of the generic goals in a process area and are a measure of the extent to which a process area has been institutionalized (performed, managed, defined) by the organization (refer to Section 5.2). Establishing capability level targets is an important element in all CERT-RMM-based improvement efforts.

When establishing capability level targets, the organization should consider the importance of the generic practices relative to the organization’s risk tolerance, threat environment, size, improvement time frame, and improvement objectives. It may be valuable to review the generic goals and generic practices and envision what the implementation of those practices and the achievement of those goals would look like for the organization during normal operations and during times of stress. Capability level targets should be established for each process area and need not be the same. Capability level 1 (performed) may be completely appropriate for a process area, even if capability level 3 (defined) is the established target for another process area in the model scope. The capability level descriptions in Section 5.3 are valuable reference material for the selection of capability level targets.

6.3.3.1 Targeted Improvement Profile

Capability level targets can be efficiently communicated in a targeted improvement profile (TIP), which is typically represented as a bar chart showing the capability level target for each process area in the model scope. Figure 6.5 provides an example of a TIP for five process areas. Figure 6.6 provides another TIP example in which fine-grained scoping options have been selected for several of the process areas. A targeted improvement profile may be integrated with a targeted improvement roadmap. In this case, the TIR may include not only the process areas selected for a specific objective, but also the TIP, which describes the capability levels that must be achieved in each process area.

Figure 6.5. CERT-RMM Targeted Improvement Profile

Figure 6.6. CERT-RMM Targeted Improvement Profile with Scope Caveats

The next section describes diagnostic methods that can be used to evaluate an organization to determine whether the capability level targets are being achieved.

6.4 Diagnosing Based on CERT-RMM

Diagnosing based on CERT-RMM is the process by which the model is used as a basis to evaluate the organization’s current resilience practices. Diagnosing can be performed formally or informally, as described in the following sections.

6.4.1 Formal Diagnosis Using the CERT-RMM Capability Appraisal Method

Formal diagnosis based on CERT-RMM is performed using the CERT-RMM Capability Appraisal Method (CAM). The CERT-RMM CAM is based on the Standard CMMI Appraisal Method for Process Improvement (SCAMPI), which has been used effectively by the CMMI community for many years [SCAMPI Upgrade Team 2006]. Similar to SCAMPI, three classes of CERT-RMM capability appraisals are available—A, B, and C—all of which are compliant with the Appraisal Requirements for CMMI (ARC) v1.2.

The class A appraisal is the most rigorous and the only one of the three methods that provides official capability level ratings. The class B appraisal has more tailoring options than class A and results in the characterization of implemented practices in the organization according to a three-point scale. Class C is even more tailorable and can be used to evaluate planned approaches to practice implementation. Some distinctions among the three methods are provided in Table 6.1.¹

Table 6.1. Classes of Formal CERT-RMM Capability Appraisals

An organization might choose a class A appraisal because it desires a rigorous examination of implemented practices that produces a rating to acknowledge or memorialize its starting point or results for an improvement project. Class A appraisals are also useful when two or more organizations are to be compared, which might be useful to evaluate different parts of a large enterprise, for example.

At the other end of the spectrum, class C appraisals are fairly lightweight and can be flexibly used to evaluate planned implementations of practices or for a less rigorous examination of implemented practices. Large organizations might choose class C appraisals to evaluate the intent of organizational policies and guidelines relative to the model. This can be an effective and efficient way to evaluate whether the resilience policies and guidelines in a large enterprise would, if followed, produce the practices that are expected in the model.

Scoping an appraisal is an important activity in planning the appraisal. The same considerations for scoping an improvement project (as described in Section 6.3.1) are used in scoping an appraisal activity. The scope of an appraisal is typically the same as the scope of the improvement effort. However, it is not required that the scope of the appraisal or other diagnostic process match the scope of the overall improvement effort. In some cases, it may be efficient to diagnose at the organizational subunit level.

Capability level ratings from class A appraisals can be shown as an overlay on the TIP diagram to clearly indicate gaps between the desired and current states, as shown in Figure 6.7. Section 6.5 provides information about analyzing and using the gaps that are identified through diagnostic activities as input to planning improvements for an organization.

Figure 6.7. Capability Level Ratings Overlaid on Targeted Improvement Profile

Appraisal results can be an important diagnostic input to inform the starting point or results of an improvement effort. Informal diagnoses can also be useful, as described in the following section.

6.4.2 Informal Diagnosis

Informal diagnosis based on CERT-RMM includes any methods, other than the formal appraisals described above, that are used to compare the organization’s practices to the guidance in CERT-RMM. Examples of informal diagnosis methods include

• meetings or tabletop exercises in which the people who are responsible for the practices in a given process area come together, review the model guidance, and discuss the extent to which the organization’s practices achieve the model intent

• reviews or analyses, supported by written reports, performed by a single person or a small group to compare the organization’s practices to the model guidance

• informal collection and review of evidence that demonstrates whether the organization is performing the model practices

In all cases, the outcomes of such diagnoses are informal findings related to the organization’s performance as compared to the model guidance. Such activities can be useful to guide informal process improvement activities or to provide information for scoping or setting capability level targets for a more formal process improvement project. Informal reviews can also be useful when the model is being used as a basis for evaluation as described in Section 6.1.2.

Both formal and informal diagnostic activities provide valuable input for planning an improvement activity. Additional considerations for planning improvement activities are described in the next section.

6.5 Planning CERT-RMM–Based Improvements

When planning a CERT-RMM deployment, analyzing gaps and determining where various practices should be optimally implemented in the organization are key activities. CERT-RMM–specific considerations related to those activities are addressed in this section.

6.5.1 Analyzing Gaps

Diagnostic activities typically reveal gaps between current and desired performance. Such gaps are necessary input to planning improvements for an organization. However, before plans are established to close any gaps that are revealed, it is important to reconsider the identified gaps in light of the overall improvement objectives. The following questions may be useful in analyzing and prioritizing the gaps in support of the improvement planning process:

• Will closing a gap support the improvement objective?

• Is the cost of closing a gap justifiable in light of the improvement objective?

• Which of the identified gaps are most important to close first?

• Can the gap be closed in one improvement iteration, or should a phased approach be deployed?

If it is determined that one or more of the identified gaps are acceptable and will not be closed, it may be appropriate to revisit and revise the objectives for the improvement activity. This iterative approach is valuable to ensure that the organization is spending improvement resources in the most productive manner. For example, if the organization chose to focus improvement efforts on a recent data breach, analyzing gaps can help the organization to prioritize improvement activities to maximize outcome at the lowest cost.

6.5.2 Planning Practice Instantiation

Part of planning improvements to existing practices or planning the implementation of new practices is to determine where in the organization the practices will be performed or instantiated. The terms organizational unit, superunit, and subunit (see Section 6.3.1) can be valuable in describing where a particular practice is to be performed in relation to the organizational scope for the improvement campaign.

Most organizations will find that different practices within a single CERT-RMM process area may be optimally performed at different levels in the organization. For example, in the Service Continuity (SC) process area, a large organization might choose to implement specific practice SC:SG1.SP2, Establish Standards and Guidelines for Service Continuity, at a very high level in the organization so that consistent standards and guidelines are established and deployed across the organization. The same organization might choose to implement specific practice SC:SG3.SP2, Develop and Document Service Continuity Plans, at a much lower level in the organization.

If the immediate or long-term improvement objective for a given process area is to achieve capability level 3, planning should include the determination of where the organizational process assets will reside. (This can be done using OPD:SG1.SP3, Establish the Organization’s Measurement Repository, and GG3, Institutionalize a Defined Process.) If the long-term plan includes a larger organizational scope than the immediate plan, then the optimal location for the organizational process assets might be different from what would be indicated by the immediate plan. Strategic consideration should be given to this issue to avoid unnecessary rework in future improvement phases. For example, Figure 6.8 shows two alternative locations for the organizational process assets in an organization. If the organization never plans to deploy CERT-RMM beyond the organizational unit shown in the figure, then either location for the organizational process assets will suffice. Suppose, however, that the organization ultimately plans to deploy CERT-RMM to the units designated by 1.1 and 1.2; in this case, the organizational process assets should be located at the highest level in the organization.

Figure 6.8. Alternate Locations for Organizational Process Assets

CERT-RMM is agile and flexible enough to support a wide range of improvement activities in an organization. The key to any successful improvement effort is to understand the objectives and to design the improvement activity to accomplish those objectives. Fine-grained scoping options are available in CERT-RMM to enable an organization to optimize the organization and model scope for an improvement. Formal and informal methods for diagnosis and comparison are available to use the model as a basis for evaluation and gap identification.