Chapter 3

Performing the Cybersecurity Risk Management Examination

Responding to Assessed Risks and Obtaining Evidence

3.01 Paragraphs .20–.21 of AT-C section 205, Examination Engagements (AICPA, Professional Standards), require the practitioner to respond to the assessed risks when designing and performing examination procedures. Specifically, they require the practitioner to

1. design and implement overall responses to address the assessed risks of material misstatement and

2. design and perform further procedures whose nature, timing, and extent are based on, and responsive to, the assessed risks of material misstatement.

3.02 Paragraph .10 of AT-C section 105, Concepts Common to All Attestation Engagements (AIPCA, Professional Standards), defines a misstatement as follows:

A difference between the measurement or evaluation of the subject matter by the responsible party and the proper measurement or evaluation of the subject matter based on the criteria. Misstatements can be intentional or unintentional, qualitative or quantitative, and include omissions. In certain engagements, a misstatement may be referred to as a deviation, exception, or instance of noncompliance.

3.03 In this guide, the following terms are used when discussing misstatements related to different aspects of the entity’s cybersecurity risk management program and the effectiveness of controls within that program:

• The term description misstatement is used when describing differences between (or omissions in) the presentation of the description of the cybersecurity risk management program and the description criteria.

• The term deficiency is used to identify misstatements in which controls were not suitably designed or did not operate effectively.

• The term deviation is used to identify misstatements in which the operation of a control was not effective in a specific instance. A deviation may, individually or in combination with other deviations, result in a deficiency.

Description misstatements and deficiencies that are immaterial do not result in a modification of the practitioner’s opinion.

Considering Materiality in Responding to the Assessed Risks and Planning Procedures

3.04 As discussed in chapter 2, “Accepting and Planning a Cybersecurity Risk Management Examination,” paragraph .16 of AT-C section 205 requires the practitioner to consider materiality when establishing the engagement strategy. Paragraph .A15 states that materiality in an attestation engagement is considered in the context of qualitative factors and, when applicable, quantitative factors. The relative importance of each of those factors when considering materiality in a particular engagement is a matter of professional judgment, and those judgments are made in light of the surrounding circumstances. Furthermore, due to the nature of a cybersecurity examination, the application of materiality to different aspects of the entity will result in differences in planned procedures due to underlying differences in threats and vulnerabilities.

3.05 Due to the vast number of information and other assets and the number of related processes and controls within even a small entity, or a business unit or segment of a larger entity, practitioners need to consider materiality during risk assessment and when determining the nature, timing, and extent of procedures to perform during the cybersecurity risk management examination. Adoption of an appropriate materiality allows the practitioner to prioritize testing efforts and supports an effective and efficient engagement.

3.06 As discussed throughout this guide, there are two distinct but complementary subject matters in a cybersecurity risk management examination:(1) the description of the entity’s cybersecurity risk management program and(2) the effectiveness of the controls within that program to achieve the entity’s cybersecurity objectives; consequently, consideration of materiality is different as it relates to each.

3.07 When considering materiality regarding the description, the practitioner should consider whether description misstatements (including omissions) in the presentation, individually or in the aggregate, could reasonably be expected to influence relevant decisions of report users. For instance, a material omission may result from the entity’s failure to describe a cybersecurity objective related to compliance with the European Union’s General Data Protection Regulation, when significant operations of the entity are subject to that regulation. Paragraph 3.19 discusses materiality considerations when evaluating whether the description is presented in accordance with the description criteria.

3.08 When considering materiality regarding the effectiveness of controls to achieve the entity’s cybersecurity objectives, the practitioner should consider both qualitative and quantitative factors, as discussed in paragraph 3.38.

Designing Overall Responses to the Risk Assessment

3.09 The assessment of the risks of material misstatement is affected by many factors, including materiality considerations and the practitioner’s understanding of the effectiveness of entity-level controls. Effective entity-level controls, particularly the control environment and monitoring activities, may allow the practitioner to have more confidence in the processes and controls the entity has designed, implemented, and operated to protect information and systems from security events that could compromise the achievement of the entity’s cybersecurity objectives and to detect, respond to, mitigate, and recover from, on a timely basis, security events that are not prevented. Thus, effective entity-level controls may reduce the nature and extent of the practitioner’s procedures to obtain evidence about control effectiveness; they may also impact decisions related to when such procedures may be performed.

3.10 In contrast, deficiencies in entity-level controls may have the opposite effect. For that reason, it is important that the practitioner understand the root cause of the deficiencies and the impact they may have on the operating effectiveness of the related controls. Ways in which a practitioner may respond to ineffective entity-level controls include

• selecting different types of procedures, or changing the timing of those procedures, to obtain evidence about the operating effectiveness of controls and

• obtaining more extensive evidence about the operating effectiveness of controls.

3.11 Paragraph .A24 of AT-C section 205 states that other overall responses to address the assessed risks of material misstatement may include the following:

• Emphasizing to the engagement team the need to maintain professional skepticism

• Assigning more experienced staff or using specialists

• Incorporating additional elements of unpredictability in the selection of procedures to be performed

• Making changes to the nature, timing, or extent of procedures

3.12 However, the importance of effective entity-level controls in a cybersecurity risk management examination go beyond providing the practitioner with more confidence in the processes and controls at the entity. For an entity with complex IT networks and architectures, effective entity-level controls may be necessary in order to establish effective internal control to achieve the entity’s cybersecurity objectives.

3.13 The remainder of this chapter discusses the nature, timing, and extent of further procedures the practitioner performs to obtain sufficient appropriate evidence in the cybersecurity risk management examination.

• Providing more supervision

Obtaining Evidence About Whether the Description of the Entity’s Cybersecurity Risk Management Program Is Presented in Accordance With the Description Criteria

3.14 As previously discussed, the description of the entity’s cybersecurity risk management program is intended to provide report users with information that will enable them to better understand the entity’s cybersecurity risk management program. For example, disclosures about the environment in which the entity operates, the process used to develop its cybersecurity objectives, commitments made to customers and others, responsibilities involved in operating and maintaining a cybersecurity risk management program, and the nature of the IT components used, allow users to better understand the context in which the processes and controls operate within the entity’s cybersecurity risk management program. Management is responsible for preparing the description and for making an assertion about whether the description is presented in accordance with the description criteria. Appendix A, “Information for Entity Management,” provides guidance to management on preparing the presentation. This section discusses the procedures the practitioner performs to obtain evidence about whether the description is presented in accordance with the description criteria.

3.15 The practitioner should obtain and read management’s description of the entity’s cybersecurity risk management program and perform procedures to determine whether the description is presented in accordance with the description criteria. The description is presented in accordance with the description criteria when it

• describes the cybersecurity risk management program the entity has implemented (that is, placed in operation);

• includes information about each description criterion presented in appendix C, “Description Criteria for Use in the Cybersecurity Risk Management Examination” of this guide; and

• does not omit or distort information that is likely to be relevant to users’ decisions. (See paragraph 3.22)

3.16 When evaluating whether the description is presented in accordance with the description criteria, the practitioner gives consideration to the implementation guidance for each criterion. The implementation guidance presents factors to consider when making judgments about the nature and extent of disclosures called for by each criterion. Because the implementation guidance does not address all possible situations, the practitioner should consider the facts and circumstances of the entity and its environment when applying the description criteria.

3.17 When considering whether the description reflects the cybersecurity risk management program and controls the entity has implemented, the practitioner should consider the understanding of the entity’s cybersecurity risk management program and controls obtained during planning, as discussed in chapter 2. The practitioner then supplements this understanding by obtaining information about the program and controls through inquiry, inspection of relevant documents, walkthroughs, and other procedures. The description is not presented in accordance with the description criteria if it (a) states or implies that aspects of the program, or controls within that program, exist when they do not or (b) inadvertently or intentionally omits information about aspects of the program or related controls that result in a presentation that could be misleading.

3.18 Management may organize its description in the manner it deems most effective, as long as each criterion is addressed within the description. Management may use various formats, such as narratives, flowcharts, tables, or graphics, or a combination thereof, to prepare the description. In addition, the degree of detail to be included in the description is generally a matter of judgment. In other words, the description is intended to be prepared at a level of sufficient detail to provide the context that users need to understand the entity’s cybersecurity risk management program; however, it is not intended to include disclosures at such a detailed level that the likelihood of a hostile party exploiting a security vulnerability is increased. Furthermore, unless specifically required by a criterion, disclosures need not be quantified.

Materiality Considerations When Evaluating Whether the Description is Presented in Accordance With the Description Criteria

3.19 Paragraph .A15 of AT-C section 205 indicates that the practitioner should consider the concept of materiality in the context of qualitative factors (as discussed in the next paragraph) and quantitative factors (for example, when management elects to disclose the percentage of time that its internet-based systems were available during the period). Accordingly, the practitioner should consider materiality when evaluating whether the description of the entity’s cybersecurity risk management program is presented in accordance with the description criteria.

3.20 As previously discussed, applying the description criteria requires judgment. One of those judgments involves the level of materiality that applies when evaluating the description of the entity’s cybersecurity risk management program in accordance with the description criteria. Because the description criteria call for disclosure of primarily nonfinancial information, most descriptions will be presented in narrative form. Thus, materiality considerations are mainly qualitative in nature and center around whether there are misstatements in, or omissions of, the information disclosed (description misstatements) that could, individually or in the aggregate, reasonably be expected to influence the decisions of intended users. For that reason, an understanding of the perspectives and information needs of intended users of the report is necessary to the assessment of materiality.

3.21 Examples of qualitative factors ordinarily considered when determining whether the description is presented in accordance with the description criteria include whether

• the description is prepared at a level of detail likely to be meaningful to report users.

• each description criterion in appendix C of this guide has been addressed without using language that omits or distorts the information related to any of the description criteria.

• the characteristics of the presentation are appropriate, since the description criteria allow for variations in presentation.

• an identified description misstatement

— is unintentional or the result of an intentional act, particularly when the person perpetrating that act is a member of management.

— is significant with regard to the practitioner’s understanding of known previous communications to report users.

— relates to the relationship between management and, if different, the engaging party or the engaging party’s relationship with other parties.

Considering Whether the Description is Misstated or Otherwise Misleading

3.22 Paragraph .60 of AT-C section 205 requires the practitioner to evaluate, based on the evidence obtained, whether the description of the entity’s cybersecurity risk management program is misleading within the context of the engagement.

3.23 When making this evaluation, paragraph .A73 of AT-C section 205 states that the practitioner may consider whether additional disclosures are necessary to supplement the description of the entity’s cybersecurity risk management program. Additional disclosures may include, for example,

• significant interpretations made in applying the criteria in the engagement circumstances (for example, what constitutes a security event or a security incident);

• subsequent events,¹ depending on their nature and significance; and

• when reporting on only a portion of the entity-wide cybersecurity risk management program, a significant security incident that occurred in another portion of that program not covered by the examination.

Such additional disclosures may be presented in the description (in which case it would be subject to the practitioner’s examination procedures) or as other information.

3.24 Although the description should be presented in accordance with the description criteria, paragraph .60 of AT-C section 205 does not require the practitioner to determine whether the description discloses every matter related to the entity’s cybersecurity risk management program that every user might consider useful when making decisions. For example, a description presented in accordance with the description criteria may omit certain information related to the entity’s cybersecurity risk management program when it is unlikely to be significant (in other words, it is immaterial) to report users’ decisions.

3.25 As part of the practitioner’s evaluation of whether the description is misleading within the context of the engagement, the practitioner may consider whether the description

• omits information involving one or more significant business units or segments, when the examination addresses the entity-wide cybersecurity risk management program.

• contains statements that cannot be objectively evaluated. For example, describing an entity as being the “world’s best” or “most respected in the industry” is subjective and, therefore, could be misleading to report users.

• contains or implies certain facts that are not true (for example, that certain IT components exist when they do not or that certain processes and controls have been implemented when they are not being performed).

• inadvertently or intentionally omits or distorts material information about any of the description criteria that might affect the decisions of report users.

3.26 If the practitioner believes that the description is misstated or otherwise misleading, the practitioner ordinarily would ask management to amend the description by including the omitted information or revising the misstated information. If management refuses to amend the description, the practitioner should consider the effect on his or her opinion about whether the presentation of the description is in accordance with the description criteria.

Evaluating the Description When the Cybersecurity Risk Management Examination Addresses Only a Portion of the Entity’s Cybersecurity Risk Management Program

3.27 As discussed in chapter 1, “Introduction and Background,” the cybersecurity risk management examination usually addresses an entity-wide cybersecurity risk management program. However, there may be circumstances in which management engages the practitioner to examine and report on only a portion of that program. In other words, the cybersecurity risk management examination may be limited to any of the following:

• One or more specific business units or segments of an entity, when those units or segments operate under an entity-wide cybersecurity risk management program

• One or more specific business units or segments of an entity, when those units or segments operate under an independent cybersecurity risk management program

• One or more specific sets of systems or particular sets of information used by the entity

3.28 In those situations, the description is tailored to disclose only information about the portion of the cybersecurity risk management program (that is, the particular business unit, segment, or type of information) within the scope of the engagement. Likewise, when evaluating whether the description is presented in accordance with the description criteria, consideration is given to whether the description addresses all relevant aspects of the portion of the cybersecurity risk management program within the scope of the engagement. For example, if the engagement addresses only one specific business unit, and that unit’s cybersecurity risk management program relies on aspects of the entity-wide program, the description would also include disclosure of those aspects of the entity-wide program relevant to that business unit.

Procedures to Obtain Evidence About the Description

3.29 Procedures the practitioner performs to obtain evidence about whether the description of the entity’s cybersecurity risk management program is presented in accordance with the description criteria include a combination of the following:

• Discussing with management and other entity personnel the content of management’s assertion and the description of the entity’s cybersecurity risk management program.

• Reading the entity’s annual report to understand

— the nature of the entity’s operations and the goods or services offered to its customers,

— the entity’s network environment and the information and systems the entity uses when interacting with customers, and

— other matters related to cybersecurity affecting financial reporting.

• Reading the entity’s cybersecurity objectives to determine whether they are suitable and complete in the specific engagement circumstances. Paragraphs 2.42 and 3.34 further discuss the suitability of an entity’s cybersecurity objectives.

• Inspecting documentation supporting the entity’s risk assessment and risk management processes, including the determination of the entity’s risk appetite and the identification and mitigation of risk.

• Reading customer contracts, performance or service-level agreements, marketing materials distributed to customers or posted on the entity’s website, and other available documentation to

— better understand the specific goods or services provided to customers and

— evaluate whether the controls the entity has implemented are suitably designed to achieve the entity’s cybersecurity objectives related to commitments to customers and others. (For example, reading service-level agreements may help the practitioner understand the specific processing commitments made, including commitments related to the timeliness of processing, expected rates of error, or persons accessing confidential information.)

• Observing control procedures or other activities performed by entity personnel.

• Reading documents (such as board minutes, organization charts, and cybersecurity communications) to understand the entity’s cybersecurity risk governance structure and processes, including

— the involvement of board members,

— the organizational structure to support the entity’s cybersecurity risk management program,

— the types of threat and vulnerability assessments the entity performs (both internal and external), and

— the types and frequency of cybersecurity communications made to executive management and others.

• Reading documents about the entity’s cybersecurity awareness and training programs, communication of code of conduct, employee handbooks, information security policies, incident notification procedures, and other available documentation to understand the entity’s processes for communicating responsibilities for cybersecurity and other related matters to entity personnel.

• Reading policy and procedure manuals, cybersecurity program documentation, flowcharts, narratives, hardware asset management records, and other system documentation to understand

— the entity’s use of technology, including its applications, infrastructure, network architecture, use of mobile devices, use of cloud technologies, and the types of external party access or connectivity to the entity;

— information technology policies and procedures; and

— controls over data loss prevention, access provisioning and de-provisioning, user identification and authentication, data destruction, security event monitoring and detection, and backup procedures.

• Reading internal audit reports, third-party assessments, audit committee presentations, and other documentation related to the entity’s cybersecurity monitoring activities, security events, or investigative activities.

• Reading example contracts with vendors and business partners (for example, contract templates or a selection of contracts) and associated performance or service-level agreements and other documentation to understand

— how the entity’s contracting process addresses cybersecurity-related matters;

— the interrelationship between the entity and its vendors and business partners, including the entity’s process for assessing and managing cybersecurity risks associated with vendors or business partners; and

— the procedures the entity performs to monitor the effectiveness of controls performed by such vendors or business partners, when such controls are material to the achievement of the entity’s cybersecurity objectives.

• Reading incident response and recovery plan documentation to understand the entity’s processes to recover from identified security events, including its incident response procedures, incident communication protocols, recovery procedures, alternate processing plans, and procedures for the periodic testing of recovery procedures.

• Reading documents describing laws, regulations, or industry standards relevant to the entity’s cybersecurity risk management program.

3.30 Performing walkthroughs provides evidence about whether the processes and controls within the program have been implemented. Performing a walkthrough involves making inquiries of management and other personnel and requesting that they describe and demonstrate their actions in performing a procedure. Walkthrough procedures include following a transaction, event, or activity from origination until final disposition through the entity’s processes, including its information systems, using the same documents and IT systems that entity personnel use. Walkthrough procedures usually include a combination of inquiry, observation, inspection of relevant documentation, and reperformance of procedures. It may be helpful for the practitioner to use flowcharts, questionnaires, or decision tables to facilitate understanding the design of the controls. An appropriately performed walkthrough provides an opportunity to verify the practitioner’s understanding of the flow of information throughout the entity’s cybersecurity risk management program and the design of the processes and controls within that program. If properly performed, walkthroughs may provide evidence about whether controls that, individually or in combination with other controls, support the key security policies and processes included in the description were implemented and operated effectively.

3.31 Inquiry, combined with other walkthrough procedures, enables the practitioner to gain a sufficient understanding of the processes and controls to determine whether they have been implemented as stated in the description of the entity’s cybersecurity risk management program. During a walkthrough, the practitioner may inquire about instances during the period in which processes and controls did not operate as described or designed. In addition, the practitioner may inquire about variations in the process for different types of information. For example, the entity’s processing may take different forms depending on how information is collected from customers or others.

3.32 In assessing whether the description is presented in accordance with the description criteria, the practitioner should consider whether there is alignment between the key security policies and processes described in the description and the controls the entity has designed and implemented to achieve the entity’s cybersecurity objectives. Although management’s description includes only information about the key security policies and processes, such key security policies and processes should be supported by controls designed and implemented to achieve the entity’s cybersecurity objectives. The lack of comprehensive alignment between the key security policies and processes included in the description and the underlying controls necessary to achieve the entity’s cybersecurity objectives would be an indicator of a description misstatement.

3.33 When performing a cybersecurity risk management examination, the practitioner should obtain an understanding of changes in the entity’s cybersecurity risk management program implemented during the period covered by the examination. If the practitioner believes that the changes would be considered significant by report users, the practitioner should determine whether those changes have been included in the description of the entity’s cybersecurity risk management program. The narrative discussing the change would be expected to contain an appropriate level of detail, including the date the change occurred and how the affected aspects of the program differed before and after the change. If such changes have not been included in the description, the practitioner may ask management to amend the description to include this information. If management refuses to include this information in the description, the practitioner should consider the effect of such changes on his or her conclusions regarding the presentation of the description of the entity’s cybersecurity risk management program and the practitioner’s opinion.

Considering the Suitability of the Entity’s Cybersecurity Objectives

3.34 As discussed in chapter 2, during the engagement acceptance process, the practitioner should consider whether management has established suitable objectives. The practitioner does not have a responsibility to express an opinion on the suitability of the entity’s cybersecurity objectives.

3.35 If, however, while performing risk assessment or further procedures, the practitioner becomes aware of information that causes him or her to believe that the cybersecurity objectives developed by management are not, in fact, suitable and complete, the practitioner should discuss the matter with management. If management is unwilling to revise the cybersecurity objectives to address the practitioner’s concerns, the practitioner should consider the effect on his or her opinion.

3.36 Assume, for example, that the client is a hospital that dispenses medication to patients through infusion pumps that are controlled through the entity’s medication system, but the client failed to establish a cybersecurity objective related to guarding against the improper use, modification, or destruction of the medication system to safeguard the life and health of its patients. Because the entity did not establish such an objective, it did not identify and assess the risks that such objective would not be achieved, nor did it design, implement, and operate controls to mitigate such risks. Accordingly, its cybersecurity objectives are incomplete and thus not suitable in the circumstances. In that situation, the practitioner may conclude that a modification of the opinion is appropriate because of the following:

• The cybersecurity objectives identified in the description in accordance with description criterion number 3 (DC3), The entity’s principal cybersecurity risk management program objectives (cybersecurity objectives) related to availability, confidentiality, integrity of data, and integrity of processing,² are not suitable; therefore, the description is not presented in accordance with the description criteria; or

• The controls were not effective to achieve the entity’s cybersecurity objectives because controls over the objective-setting process are ineffective based on the entity’s failure to meet control criterion 3.1 (CC3.1), The entity specifies objectives with sufficient clarity to enable the identification and assessment of risks relating to objectives.

3.37 Because the entity’s cybersecurity objectives need to be suitable to enable both management and the practitioner to evaluate whether internal control over cybersecurity is effective, the lack of suitable cybersecurity objectives is likely to have a pervasive effect on the effectiveness of the entity’s cybersecurity risk management program. Accordingly, it is likely that the practitioner would express an adverse opinion on both subject matters.

Materiality Considerations When Evaluating the Effectiveness of Controls to Achieve the Entity’s Cybersecurity Objectives

3.38 Paragraph 3.19 discusses materiality considerations related to the description, whereas this section discusses materiality considerations that can affect the practitioner’s conclusion about the suitability of design and operating effectiveness of controls to achieve the entity’s cybersecurity objectives.

3.39 When considering whether controls within the program were effective to achieve the entity’s cybersecurity objectives, the practitioner should consider a number of factors, including

• the nature of threats, and the likelihood and magnitude of the risks arising from those threats, to specific information assets.

• the technical environment, including whether the realization of those threats or the exploitation of vulnerabilities related to specific information assets that appear inconsequential could expose (either directly or indirectly) the information assets and result in controls that were not effective to achieve the entity’s cybersecurity objectives. For example, if access to the information assets of a financially immaterial business unit could provide access to the entity’s strategic business systems, and the practitioner determines there is a high likelihood that such a vulnerability might be exploited, the practitioner is likely to consider access to the information assets of the financially immaterial business unit to be material in the cybersecurity risk management examination.

• the nature of threats arising from error or fraud, and the likelihood and magnitude of the risks arising from such threats, to the operation of processes and controls that support the achievement of the entity’s cybersecurity objectives, and the vulnerabilities of those processes and controls to those threats. For instance, the security operation center staff’s lack of knowledge regarding new types of cyberattacks may result in the failure to detect, in a timely manner, a security incident that could significantly affect the entity’s achievement of its cybersecurity objectives; consequently, this deficiency could result in a material failure to achieve a cybersecurity objective.

3.40 The practitioner should consider both qualitative and quantitative factors when evaluating control effectiveness. Qualitative factors the practitioner considers include the following:

• Relevance of a control to achieve a particular cybersecurity objective based on the control criteria. Not all controls that have been implemented need to be considered if the control criteria are met through the application of other controls. As an example, assume an entity mirrors data to a data center located in another city and creates tapes of the data as a secondary backup. These tapes are stored at a third location. Data written to the backup tapes is encrypted. The entity has identified the encryption of the tape as a control; however, the entity has not identified physical security controls over the tape storage location in its description because management concluded that

— the risk that both the primary data center and the mirror site are destroyed simultaneously is remote and

— encryption of the data on the tapes is sufficient to achieve the entity’s cybersecurity objectives with regard to protecting the confidentiality of the information based on the control criteria.

In this example, physical access controls over the tape storage location are unlikely to be material or relevant because controls over the encryption of the tapes prevent unauthorized access.

• Alignment between the key security policies and processes included in management’s description and the underlying controls within the entity’s cybersecurity risk management program. If manage-ment’s description includes a particular system in the entity operations summary or in the listing of information assets, it is likely that report users would presume that system is material for the purposes of the cybersecurity risk management examination. Similarly, report users are likely to expect that controls that, individually or in combination with other controls, support the key security policies and processes described in management’s description would ordinarily be tested and evaluated as part of the evaluation of control effectiveness.

• Practitioner’s understanding of previous communications made to report users regarding cybersecurity. If the practitioner becomes aware that the entity has made representations to report users regarding cybersecurity (for instance, through a presentation on the entity’s website that indicates that all client data is kept encrypted at all times), the practitioner is likely to consider those representations important to such users.

• Relevance to compliance with laws and regulations. If the entity is subject to requirements specified by laws or regulations related to cybersecurity, identified deficiencies and deviations related to compliance are likely to be significant since they may have additional consequences to the organization. Requirements established by laws and regulations may therefore need to be included in the consideration of materiality and the related engagement strategy. For laws and regulations that have a direct effect (for example, laws protecting sensitive personal health information), the entity may establish cybersecurity objectives regarding compliance with such laws. Other laws and regulations may be less directly linked to the cybersecurity objectives but may still be relevant to the examination (for example, regulations over the physical storage of biohazard materials, when the materials are stored in a warehouse with access secured by an electronic badging system).

• Interactions with third parties. Materiality considerations are based on factors such as the likelihood and magnitude of cybersecurity risks arising from interactions with third parties (customers, vendors, business partners, or others) with access to the entity’s system, the degree to which those risks are relevant to the entity’s cybersecurity risk management program, and the extent to which the entity monitors controls performed by those third parties.

• Indicators of the operating effectiveness of cybersecurity performance activities. Indicators of the operating effectiveness of control activities, such as the number and nature of security events resulting in a loss, the mean time from first occurrence to detection, and the mean time from detection to remediation, may be indicative of challenges in the design or operating effectiveness of cybersecurity controls; accordingly, such factors may affect materiality judgments.

• Degree to which controls are designed to identify and address threats and vulnerabilities that are currently unknown. Certain controls may have the ability to detect and address unknown threats. An example of this is a data loss prevention (DLP) control that monitors and restricts outbound information, regardless of what caused the attempt to send the information externally.

• Threats related to prior periods. An identified threat or vulnerability in a prior period may affect the assessment of the entity’s cybersecurity risk management program or the effectiveness of controls for the current period.

• Effect of deviations. Identified deviations may affect the entity’s ability to mitigate threats or vulnerabilities to information and other assets and achieve the related cybersecurity objectives. For example, the practitioner may question management’s assertion that a control is effective when considering the nature and extent of observed deviations in the operation of the control.

• Intentional acts. A deficiency or deviation may be the result of an unintentional act or may be intentional. An intentional act perpetrated by management or senior management would be particularly relevant to materiality considerations.

• Relationship to other parties. A deficiency in controls may relate to the relationship between the entity and other parties. For example, a deficiency in controls at the entity that could also result in a deficiency in controls at a customer is more likely to be considered material.

3.41 Quantitative factors to be considered in a cybersecurity risk management examination relate to matters such as the tolerable rate of deviation and the observed rate of deviation. (In this guide, the tolerable rate of deviation is the maximum rate of deviation in the operation of the control that the practitioner is willing to accept without modifying the opinion relating to one or more of the control criteria.) Quantitative factors are less likely to apply when evaluating the design of controls but would be considered when evaluating the operating effectiveness of the controls. Note, however, that the practitioner should carefully consider the effect of identified deviations, either individually or in combination with other identified deviations, on the controls’ ability to mitigate assessed risks because such deviations could result in the failure to achieve one or more of the entity’s cybersecurity objectives.

3.42 Paragraph .17 of AT-C section 205 indicates the practitioner should reconsider materiality if the practitioner becomes aware of information during the engagement that would have caused him or her to have initially determined a different materiality.

Obtaining and Evaluating Evidence About the Suitability of the Design of Controls to Achieve the Entity’s Cybersecurity Objectives

3.43 As discussed in chapter 1, the practitioner’s opinion on the effectiveness of controls encompasses both the suitability of the design of controls and their operating effectiveness. Because there are specific considerations when evaluating each, this chapter contains separate discussions of suitability of design and operating effectiveness to support the overall opinion on the effectiveness of controls to achieve the entity’s cybersecurity objectives. This section discusses evaluating the suitability of design, whereas the section beginning in paragraph 3.57 discusses evaluating the operating effectiveness of controls.

3.44 Paragraph .15 of AT-C section 205 states that the practitioner’s understanding of the controls within an entity’s cybersecurity risk management program includes an evaluation of the design of controls within that program and whether they have been implemented. Suitably designed controls, if complied with satisfactorily, provide reasonable assurance of achieving the entity’s cybersecurity objectives based on the control criteria. Suitably designed controls operate as designed by persons who have the necessary authority and competence to perform the controls.

3.45 Matters that are relevant in determining whether controls are suitably designed include the following:

• Whether the applicable control or set of controls adequately addresses the risks that threaten the achievement of the entity’s cybersecurity objectives based on the control criteria

• Whether the applicable control or set of controls, if operated effectively, would protect information and systems from security events that could compromise the achievement of the entity’s cybersecurity objectives and detect, respond to, mitigate, and recover from, on a timely basis, security events that are not prevented

• Whether the information used in the operation of the controls is reliable. For example, the operation of a control may rely on configuration parameters of the comparison of the data to another set of data that is expected to be complete and accurate.

• Whether the applicable control or set of controls is adequately changing, adapting, and evolving, from a cyber-threat monitoring perspective, as new threats and exploits are identified and become able to be defended against by entities.

3.46 Management is responsible for designing and implementing controls to achieve the entity’s cybersecurity objectives, identifying the risks that threaten the achievement of the objectives, modifying the controls as necessary based on new and evolving risks, and evaluating the linkage between the controls and the evolving risks and threats that threaten the achievement of the objectives. In many cases, the practitioner is able to obtain management’s documentation of its identification of risks and evaluation of the linkage of controls to those risks. In these instances, the practitioner may evaluate the completeness, accuracy, relevance, and timeliness of management’s identification of risks and the design of the controls in mitigating those risks. The practitioner may also contemplate whether the controls designed and implemented by management achieve the cybersecurity objectives based on the current operating environment, the known risks and threats as of a given point in time, and the exploitation of evolving vulnerabilities.

3.47 When considering the suitability of design, the practitioner should also consider (a) management’s process for assessing risks and for designing and implementing controls to address those risks, (b) the results of walk-throughs, and (c) evidence about the operating effectiveness of controls that indicated a deficiency in the design of the controls, in light of the practitioner’s knowledge and experience and the particular circumstances. Controls are intended to mitigate the risks that the entity’s cybersecurity objectives will not be achieved. For example, the risk that a server will not be able to support availability in the event of a distributed denial of service attack can be addressed by a control that provides redundant load balanced infrastructure protected by mechanisms for detecting and dropping access attempts.

3.48 Identified risks that may impact the achievement of the entity’s cybersecurity objectives also encompass fraud such as management’s override of identified controls at the entity, misappropriation of assets by entity personnel, creation by entity personnel of false or misleading documents or records, and inappropriate physical and logical access controls to information and the underlying infrastructure through social engineering attacks or similar measures. The practitioner should consider both the risk of fraud and errors in evaluating the suitability of the design of controls.

3.49 The practitioner’s evaluation of management’s risk assessment process (that is, the assessment of potential events and circumstances that could threaten the achievement of the entity’s cybersecurity objectives) includes consideration of items such as the following:

• The process management uses to

— identify its cybersecurity objectives,

— identify information and other assets,

— determine the threats to information and other assets,

— design and implement controls to address identified risks, and

— incorporate information from its monitoring activities that identify previously unconsidered potential events and circumstances

• The frequency with which management updates the risk assessment and supporting risk management processes and controls

• Whether management uses an appropriate management framework for managing its processes and controls (for example, the National Institute of Standards and Technology “Frame-work for Improving Critical Infrastructure Cybersecurity” [NIST cybersecurity framework] or International Standardization Organization/International Electrotechnical Commission [ISO/IEC] Standards 27001 and 27002) as part of its assessment and management process

3.50 Factors such as the size and complexity of the entity, the goods or services provided, and commitments made to customers and others are important considerations when evaluating the suitability of the design of controls. A smaller, less complex entity may be able to address risks that threaten the achievement of the entity’s cybersecurity objectives using a different set of controls than a larger, more complex entity. For example, a smaller, less complex entity may

• have policies and procedures that are less formal and detailed but sufficient for the practitioner to evaluate;

• have fewer levels of management, which may result in more direct oversight of the operation of key controls; and

• make greater use of manual controls versus automated controls.

3.51 When considering suitability of design, the practitioner may determine that some information assets (such as network access points, databases, or transactions) are subject to greater threats or have vulnerabilities that are more likely to be exploited. In such instances, control activities designed and implemented to prevent or detect security events associated with these threats and vulnerabilities may require greater precision and reliability in order to be suitably designed.

3.52 The practitioner evaluates the suitability of the design of controls by using evidence and other information obtained when

• obtaining an understanding of the cybersecurity risk management program and the controls within that program;

• determining whether the description of the entity’s cybersecurity risk management program was presented in accordance with the description criteria (including evidence obtained from performing walkthroughs); and

• performing a combination of the following procedures:

— Inquiry of entity personnel regarding the design and operation of applicable controls and the types of security events that have occurred or that may occur

— Inspection of documents produced by the entity

— Performing additional walkthroughs of control-activity-related policies and procedures

— Reading applicable and supporting program documentation

— Determining whether attacks and vulnerability exploitations, including those that are well established in the hacker community as well as emerging risks and threats, are addressed to achieve the entity’s cybersecurity objectives

3.53 To evaluate the suitability of the design of the controls within the entity’s cybersecurity risk management program, the practitioner should consider the following information about the controls:

• The frequency or timing of the occurrence or performance of the control

• The authority and competence of the individual responsible for conducting the activity (for example, details regarding the appropriateness of the level of the individual performing the control, their role in the organization, and conflicting duties).

• The tasks within the activity being performed and the precision and sensitivity of those tasks (for example, the results of reviews and related follow-up activities)

• Contrary evidence that the control is not functioning as designed, such as the rate of security incidents identified related to the control

3.54 After performing the procedures and considering the guidance in paragraphs 3.38–3.42, the practitioner should consider whether the controls have the ability, as designed, to provide reasonable assurance of achieving the entity’s cybersecurity objectives based on the control criteria. Further, the practitioner should consider whether the appropriate controls are in fact in place given the circumstances.

Identifying and Evaluating Deficiencies in the Suitability of Control Design

3.55 In determining whether there is a deficiency in the design of a control, the practitioner determines whether

• a control necessary to meet the one or more control criteria is missing or

• an existing control is not properly designed, meaning that, even if the control operates as designed, one or more control criteria would not be met.

3.56 When evaluating the suitability of the design of controls, the practitioner determines whether the controls are appropriate and whether they have been implemented. If a necessary control does not exist, this would be considered a design deficiency. If deficiencies exist in the design of a control, the practitioner often would not test the operating effectiveness of that control. Rather, the practitioner generally would consider the design of other controls that address the same risks.

Obtaining Evidence About the Operating Effectiveness of Controls to Achieve the Entity’s Cybersecurity Objectives

3.57 Controls are suitably designed if they have the potential to achieve the entity’s cybersecurity objectives based on the control criteria. Suitably designed controls are operated as designed by persons who have the necessary authority and competence to perform the control. Controls that operated effectively provide reasonable assurance of achieving the entity’s cybersecurity objectives based on the control criteria.

3.58 A control may be designed to address an identified risk on its own or may function in combination with other controls. For example, when a supervisor, prior to approving user credentials, is reviewing the list of authorized users to determine whether a new user has been authorized by the entity to access one or more of its systems, the review control (reviewing and approving the user’s credentials) may be complemented by an application control requiring that the supervisor acknowledge his or her review and approval by entering a sign-off in the system. In this instance, both the manual and automated controls would be tested by the practitioner because the two controls are dependent on each other.

3.59 The practitioner should obtain information from management regarding changes made to controls during the period covered by the practitioner’s report. In addition, during the performance of his or her procedures, the practitioner is alert for any changes that may not have been identified by management. If the practitioner believes the control changes could be significant to users of the report and could be relevant to meeting one or more of the control criteria, both the superseded controls and the updated controls would be included in the controls the practitioner would test.

Designing and Performing Procedures to Evaluate the Operating Effectiveness of Controls

3.60 Paragraph .24 of AT-C section 205 requires the practitioner to design and perform tests of controls to obtain sufficient appropriate evidence about the operating effectiveness of controls. The practitioner is responsible for determining the nature (how the controls are tested), timing (when the controls are tested and the frequency of the testing), and extent (the number of testing procedures performed or size of the sample) of testing necessary to provide sufficient and appropriate evidence that the controls operated effectively throughout the specified period of time.³

3.61 When determining the nature, timing, and extent of procedures to be performed to obtain sufficient appropriate evidence of the operating effectiveness of controls, the practitioner should consider the type of evidence that can be obtained from the performance of the control and how long that evidence will be available.

3.62 If the practitioner determines that certain entity-level controls (control environment, communication and information, risk assessment, and monitoring controls) did not operate effectively, the practitioner may be able to adjust the nature, timing, and extent of procedures performed to obtain evidence about whether the entity’s controls were effective to achieve the entity’s cybersecurity objectives. In some situations, deficiencies in the operation of entity-level controls may lead the practitioner to conclude that controls are not operating effectively to achieve certain cybersecurity objectives. For example, consider an entity whose ability to retain knowledgeable employees has been impaired. The practitioner may decide to increase the testing of controls that prevent and detect security incidents (for example, inspection of security configurations and event management scan logs) to determine whether controls operated effectively to achieve the cybersecurity objectives based on the control criteria.

Nature of Procedures to Evaluate the Effectiveness of Controls

3.63 When designing and performing tests of controls, the practitioner

1. makes inquiries and performs other procedures to obtain evidence about the following:

   1. How the control was implemented (For example, was the control performed as designed?)

   2. The level of consistency with which the control was applied throughout the period

   3. By whom or by what means the control was applied (For example, is the control automated or manual? Has there been high turnover of personnel in the position that performs the control? Is the control being performed by an inexperienced person?)

2. determines whether the controls to be tested depend on other controls and, if so, whether it is necessary to obtain evidence supporting the operating effectiveness of those other controls.

3. determines an effective method for selecting the items to be tested to meet the objectives of the procedure.

3.64 Other procedures that the practitioner performs, in combination with inquiry, to obtain evidence about the operating effectiveness of controls include one or more of the following:

• Observation of the application of the control

• Inspection of documents, reports, or electronic files that contain evidence of the performance of the control, such as system log files

• Reperformance of certain controls performed by management, such as access recertification and security event log reviews

3.65 Because of the nature and methods of information storage used in the operation of cybersecurity control activities, the practitioner may find the use of analytics to be a highly effective technique in performing his or her procedures, such as in the following examples:

• Documentation of authorization of approvals of management may be stored in an online workflow system permitting the records from the system to be extracted and analyzed.

• System logs may be scanned for unusual activity.

• Server security configuration parameters may be scanned and analyzed for consistency with policy.

• Access control lists can be analyzed for appropriateness of access rules.

When using analytics, the practitioner would perform procedures to validate the completeness and accuracy of the information received from the entity.

3.66 Inquiry alone does not provide sufficient appropriate evidence of the operating effectiveness of controls. Some procedures provide more convincing evidence of the operating effectiveness of controls than others (for example, inquiry combined with inspection or reperformance ordinarily provides more convincing evidence than inquiry and observation alone).

3.67 The type of control being tested may affect the nature, timing, and extent of the testing performed by the practitioner. For example, for some controls, operating effectiveness is evidenced by documentation. In such circumstances, the practitioner may inspect the documentation. Other controls may not leave evidence of their operation that can be tested at a later date, and accordingly, the practitioner may need to test the operating effectiveness of such controls at various times throughout the specified period via observation.

3.68 Evidence of the operating effectiveness of a control may be lost, misplaced, or inadvertently deleted by the entity. In such instances, the practitioner determines whether other evidence of the operating effectiveness of the control exists and whether the results of tests would provide sufficient appropriate evidence. If not, the practitioner should consider whether there are other effective controls in place to achieve the entity’s cybersecurity objectives based on the control criteria. If certain limitations exist in the ability to retain evidence (such as security logs), the practitioner may plan to obtain such evidence at multiple intervals throughout the examination period.

3.69 In addition to procedures to directly test the operation of a control, the practitioner may also perform procedures to obtain evidence about whether the control functioned to prevent or detect errors and fraud. For example, when testing the effectiveness of an entity’s vulnerability scanning controls, the practitioner may use his or her own vulnerability scanning tool to detect unidentified vulnerabilities in order to assess the effectiveness of the entity’s controls. As another example, the practitioner might obtain a listing of the security incidents identified during the period and compare the vulnerabilities exploited to the controls implemented to protect information and other assets in order to identify deficiencies in the design or operation of the related control activities.

Evaluating the Reliability of Information Produced by the Entity

3.70 When using information produced by the entity, paragraph .35 of ATC section 205 requires the practitioner to evaluate whether the information is sufficiently reliable for the practitioner’s purposes, including, as necessary, the following:

1. Obtaining evidence about the accuracy and completeness of the information

2. Evaluating whether the information is sufficiently precise and detailed for the practitioner’s purposes

3.71 Examples of information produced by the entity’s information system include the following:

• Population lists the practitioner uses to select a sample of items for testing

• Manually prepared or system-generated reports

• Exception reports generated by the system

• Ad hoc request reports

• Documentation that provides evidence of the operating effectiveness of controls, such as user access lists

• Logs from security tools (for instance, data loss prevention, network activity, vulnerability scans)

3.72 The results of the practitioner’s tests will not be reliable if the population from which the items have been selected for testing is incomplete. As an example, the effectiveness of a control, such as the periodic review of user access, is affected by the completeness and accuracy of the information used to prepare the user access reports. In this situation, the practitioner would inspect the scripts used to create user access reports for accuracy of logic.

3.73 The practitioner identifies the information produced by the entity while performing procedures to assess the design, implementation, and operating effectiveness of controls within the entity’s cybersecurity risk management program. When assessing the information produced, the practitioner should consider the reliability of the information, specifically the completeness and accuracy of the information. For example, if the practitioner intends to test a population of user terminations during the period under examination, the practitioner would perform procedures to determine that the lists of terminated users generated from human resource management systems are complete and accurate.

3.74 The information may be produced only once or on a recurring basis for use in the execution of a control. The information may be produced manually by management or generated from a system. When the information produced by the entity is provided to the practitioner, the practitioner assesses how the information is used, the source of the information, and the impact the information could have on the engagement.

3.75 Depending on the means by which the practitioner obtains the information, the practitioner would develop a plan to assess the completeness and accuracy of the data. The information may also provide evidence of the operating effectiveness of a control. When assessing information used in the execution of controls, the practitioner should consider the following factors:

• The level of assurance being sought from the control

• The degree to which the effectiveness of the control depends on the completeness and accuracy of the information

• The precision with which the control is performed (for example, precision of review controls)

• The degree to which the control depends on other controls

3.76 Additional items to be considered by the practitioner when assessing the completeness and accuracy of information may include the following:

• Where is the information produced or generated from—the entity’s applications or systems, other sources, or third parties?

• Is the information located in a controlled information technology environment or an ad hoc reporting database or data warehouse?

• Is the information highly structured and complex or relatively straightforward?

• What is the basis for the entity’s comfort regarding the completeness and accuracy of the data or information?

3.77 Determining the nature and extent of evidence needed to assess the completeness and accuracy of data is a matter of professional judgment. When obtaining evidence about the completeness and accuracy of the information, the practitioner may perform this as part of his or her tests of the effectiveness of controls or may develop specific procedures to be applied to the information received. The more important the control or information, the more persuasive the evidence needed about the completeness and accuracy of the information. In addition, the practitioner should considers the need to ascertain the completeness and accuracy of the information throughout the period covered by the cybersecurity risk management examination.

3.78 The following are examples of procedures the practitioner may perform when the information being tested has been produced by the entity:

• Example 1 (Population of incidents). The incident management recordkeeping application generates a report of all incidents during a period. Before testing a sample of such incidents, the practitioner may inspect the query logic used to generate the report and perform a walkthrough of the process used to record incidents in the application. The practitioner may also inspect the report for anomalous gaps in sequence or timing to determine completeness.

• Example 2 (Population of changes). The change management system is used to communicate changes ready for implementation. Before testing a sample of changes to application software, the practitioner may perform a walkthrough of the process used to communicate changes ready for implementation in order to understand whether any alternate paths of communication exist. The practitioner would also assess the segregation of duties between those responsible for the development and testing of the changes and those responsible for migration of changes to the production environment. The practitioner would also consider the enforcement of the segregation of these duties through logical access controls.

• Example 3 (Population of servers). All servers are included in vulnerability scans. Before testing the results of a sample of vulnerability scans, the practitioner would ascertain the process for performing the vulnerability scans (for example, subnet scanning, manually adding server names) and the configurations used to include the entity’s relevant environments. The practitioner would need to understand and consider how the server build-out process is conducted and how servers are migrated to the relevant environments to be included in the scanning.

Timing of Procedures

When the Examination is for a Specified Period of Time

3.79 When the examination is for a period of time specified by management, the practitioner should obtain evidence about the operating effectiveness of controls over the period of time covered by the examination to support the opinion. Based on consideration of a number of factors, the practitioner may decide to perform procedures at interim dates, at the end of the examination period, or after the examination period, when evidence of the operation of controls during the period is available after the end of the period. The following are some relevant factors to consider when determining the timing of procedures:

• The nature of the controls

• The period of time during which the information will be available (for example, electronic files may be overwritten after a period of time or hard copy records may not be retained)

• Whether testing requires direct observation of a procedure that is only performed at certain times during the examination period

• Whether the control leaves evidence of its operation and, if not, whether the control must be tested through observation

3.80 Performing procedures at an interim date and communicating deviations and deficiencies to management at an early stage in the examination may provide management with an opportunity to make changes in the design or operation of controls to correct the deviations for the remaining portion of the examination period.

3.81 When the practitioner performs tests of the operating effectiveness of controls at an interim period, the practitioner should determine what additional testing is necessary for the remaining period.

When the Examination is as of a Point in Time

3.82 When the practitioner is reporting as of a point in time, the practitioner should obtain sufficient appropriate evidence about the operating effectiveness of controls. Because not all controls will operate on a daily basis, the practitioner should consider the timing of his or her procedures over an appropriate interval prior to the point in time at which effectiveness is evaluated. Furthermore, a single instance of the operation of a control may not provide sufficient appropriate evidence that a control is operating effectively. In establishing the nature, timing, and extent of procedures to be performed when reporting as of a point in time, the practitioner determines the timeframe over which the operation of a control can be tested to support his or her conclusion as of the specified point in time, as well as the number of instances of the operation of the control necessary to conclude on the effective operation of the control. All such decisions are a matter of professional judgment.

Extent of Procedures

3.83 The practitioner should design and perform tests of controls and other procedures to obtain sufficient appropriate evidence that controls operated effectively throughout the period to achieve the entity’s cybersecurity objectives based on the control criteria. Relevant factors in determining the extent of tests of controls include the following:

• The nature of the controls

• The frequency of the performance of the control during the period (for example, daily management review of open incidents versus monthly review of closed incidents to identify ongoing problems)

• The relevance and reliability of the evidence that can be obtained to support the conclusion that the controls operated effectively to meet the control criteria

• The extent to which evidence is obtained from tests of other controls designed to meet the same criterion

3.84 The practitioner should obtain evidence about the operating effectiveness of controls throughout the examination period. In some cases, however, a control may not operate frequently enough to be assessed as operating effectively. For example, if a control operates only annually in December, and the examination covers the six-month period from January 1, 20XX, to June 30, 20XX, the practitioner is unable to test the operating effectiveness of that control throughout the period. In other instances, a control may not operate because the circumstances that trigger its operation do not occur during the period covered by the examination. The latter situation is discussed further beginning in paragraph 3.99.

3.85 The shorter the test period, the greater the risk that controls may not have operated effectively throughout the period or that the practitioner will be unable to obtain sufficient evidence to express an opinion on the operating effectiveness of those controls. For example, testing the operation of a monitoring activity for only a limited portion of the examination period may not be indicative of the associated control’s effectiveness throughout the period. Depending on the significance of the controls to the achievement of the entity’s cybersecurity objectives based on the control criteria, the practitioner may decide to express a qualified opinion or disclaim an opinion because of the limitation on the scope of the engagement.

3.86 When evaluating the operating effectiveness of controls, the practitioner may consider the results of tests performed while providing other services to the entity. Furthermore, deviations in the operation of a control identified during the prior year’s examination may impact the practitioner’s risk assessment for that control, which may cause the practitioner to increase the extent of testing in the current period. For example, if the practitioner’s opinion in the prior year was qualified because of deficiencies in controls over the authorization of user access due to the inexperience of the person performing the controls, the practitioner may decide to increase the number of items tested in the current examination period to determine if the deficiency has been effectively corrected.

3.87 An automated control usually functions consistently unless the program, including the tables, files, or other permanent data used by the program, is changed. Once the practitioner determines that an automated control is functioning as intended, which could be determined at the time the control is initially implemented or at some other date, the practitioner should perform tests to determine that the control continues to function effectively. Such tests ordinarily would include determining that changes to the program are not made without being subject to the appropriate program change controls, that the authorized version of the program is used for processing transactions, and that other relevant controls are effective.

3.88 If a control operates frequently, the practitioner may consider whether to use audit sampling when testing the operating effectiveness of the control. When determining the extent of tests of controls and whether sampling is appropriate, the practitioner should consider (a) the characteristics of the population of the controls to be tested, including the nature of the controls, (b) whether the population is made up of homogenous items, (c) the frequency of their application, and (d) the expected deviation rate. The AICPA Audit Guide Audit Sampling may be useful to the practitioner when performing sampling.

3.89 Before deciding to use sampling in a cybersecurity risk management engagement, the practitioner should consider whether sampling is an appropriate strategy for testing the control. For example,

1. due to the design of one or more systems, it may not be possible to give every item in the population a chance of being selected for the sample.

2. the practitioner may determine that a 100 percent test of the control using data analytics is necessary because even a one-time failure of the control could result in failure to achieve the entity’s cybersecurity objectives.

3. the practitioner may conclude that it is more efficient and more effective to perform a 100 percent test of the data evidencing the effective operation of the control than selecting and testing a sample.

In such circumstances, sampling may not be an appropriate approach to obtaining sufficient appropriate evidence to evaluate the effectiveness of the control. Consequently, in applying professional judgment regarding the extent of testing, the practitioner needs to consider whether the assumptions for sample-based testing have been met.

Selecting Items to Be Tested

3.90 For tests of controls using sampling, the practitioner determines the tolerable rate of deviation and uses that rate to determine the number of items to be selected for a particular sample.

3.91 The practitioner’s selection of sample items should be reasonably expected to be representative of the population, resulting in a sample that is representative of the population covering the reporting period. Random-based selection of items represents one means of obtaining such samples.

Testing Changes to Controls

3.92 If, during the examination period, the entity makes changes to controls that are relevant to achieving the entity’s cybersecurity objectives based on the control criteria, the practitioner should test, if possible, the superseded controls before the change and test the new controls after the change for the period they were in effect and consider whether the change in control was appropriately addressed in the description of the system. For example, assume that during the examination period June 1, 20X0, to May 31, 20X1, the entity automated a control that was previously performed manually. If the entity automated the control on December 15, 20X0, the practitioner would test the manual control for the period from June 1, 20X0, to December 14, 20X0, and test the automated control for the period from December 15, 20X0, to May 31, 20X1. If the practitioner cannot test the superseded controls (for example, because the controls did not leave evidence of operation after a period of time or the practitioner was engaged after the controls were superseded), the practitioner should determine the effect on the practitioner’s report.

Risk Mitigation and Control Considerations Related to Third Parties

3.93 Given the prevalence and ease with which information, operations, and processes are shared and exchanged across traditional organizational boundaries, an entity needs to carefully consider the cybersecurity risks posed by interactions with third parties.

3.94 As discussed in chapter 2, third parties include customers, vendors, business partners, and others with access to one or more of the entity’s information systems who store confidential entity information on their systems, or who otherwise transmit information back and forth between themselves and the entity, or on behalf of the entity. Consider the following:

• Vendor performs cybersecurity processes and controls. An entity may engage a service provider to perform cybersecurity processes and controls. Examples of such processes and controls include the following:

— The performance of periodic vulnerability scans, penetration tests, and other critical monitoring activities;

— The deployment of proprietary cybersecurity breach detection sensors throughout the entity’s IT network and the monitoring and investigation of security events detected by those sensors; and

— The preparation of and reporting of customer analytics related to the entity’s system

• Vendor accesses entity’s information assets. An entity may permit a vendor to access its raw materials inventory system and production schedules in order to time the delivery of shipments of production inputs.

This access to the entity’s systems by a third party gives rise to additional vulnerabilities to the entity’s IT systems that could be exploited and result in controls that are not suitably designed to achieve one or more of the entity’s cybersecurity objectives.

3.95 In response to such risks from third parties, management needs to understand the nature of the cybersecurity risks posed by the third parties, assess the likelihood and magnitude of such risks, and design and implement monitoring controls to address those risks. Management also needs to recognize and acknowledge that, even after implementing its strategy, management will be dependent on the cybersecurity risk management and control activities of the third party. For this reason, among others, it is important that management participate in ongoing communications with third parties to discuss changes to the third parties’ processes and controls as the need arises.

3.96 The entity’s cybersecurity risk management program ordinarily includes procedures to properly identify and assess the cybersecurity risks posed by third parties and to implement monitoring controls to address those risks. Such procedures and controls are commonly included in a third-party risk management program. Among other things, a third-party risk management program often includes procedures to obtain evidence about the effectiveness of the third party’s processes and controls.

3.97 When determining the nature, timing and extent of procedures to obtain evidence about whether the entity’s monitoring controls over the third party’s processes and controls were effective in the circumstances, the practitioner’s procedures ordinarily will depend upon the nature and extent of the entity’s monitoring controls. For example, if the entity has obtained a type 2 SOC 2 report on aspects of a third party’s operations that relate to the processing integrity of its services, as well as its security, availability, and confidentiality controls, the practitioner might review the report to determine whether management has adequately evaluated it by assessing (a) the relevance of the system description and complementary user entity controls to its own cybersecurity risk management program and (b) any deviations requiring further evaluation and response by management. If the third party does not provide management with a type 2 SOC 2 report, management may perform direct testing of the third party’s controls by obtaining evidence from that party of the effectiveness of its controls. However, unless the practitioner is reperforming management’s tests of the third party’s controls, the practitioner’s performance of tests directly on the third party’s controls would not provide evidence about the effectiveness of the entity’s cybersecurity controls. In any event, the practitioner should obtain sufficient appropriate evidence of the effectiveness of the third party’s controls. In addition, the practitioner needs to consider whether the third party’s use of its own IT system and connections to the entity’s IT network and assets represents new vulnerabilities that need to be assessed and addressed as part of the entity’s third-party risk management program.

3.98 When evaluating the effectiveness of the controls within the entity’s cybersecurity risk management program, the practitioner needs to conclude on whether the entity’s monitoring controls over the processes and controls performed by third parties are effective to achieve the entity’s cybersecurity objectives. When the practitioner is unable to reach such a conclusion, or when the practitioner determines that such activities were ineffective to achieve the entity’s cybersecurity objectives, the practitioner’s report should be modified accordingly.

Controls Did Not Need to Operate During the Period Covered by the Practitioner’s Report

3.99 Management’s description of the entity’s cybersecurity risk management program includes, among other things, a description of the key security policies and processes that ordinarily operate during the period covered by the cybersecurity risk management examination report. In some cases, however, the circumstances that trigger the operation of certain of those processes do not occur; therefore, some or all of the related controls do not operate during the period covered by the cybersecurity risk management examination report. For example, if no identified security incidents required the recovery of systems, data, or other information assets, the recovery controls (such as restoring systems and data from clean backups and replacing compromised files) would not operate. When management informs the practitioner that the events requiring the operation of a control did not occur, the practitioner should obtain sufficient appropriate evidence to corroborate management’s statement. Reporting in this situation is discussed in chapter 4, “Forming the Opinion and Preparing the Practitioner’s Report.”

Revising the Risk Assessment

3.100 Paragraph .34 of AT-C section 205 clarifies that the practitioner’s assessment of the risks of material misstatement may change during the course of the engagement as additional evidence is obtained. In circumstances in which the practitioner obtains evidence from performing further procedures or when new information is obtained, either of which is inconsistent with the evidence on which the practitioner originally based the assessment, the practitioner should revise the risk assessment and modify the planned procedures accordingly. This may require the performance of additional procedures as necessary.

Using the Work of Internal Auditors

3.101 Chapter 2 of this guide discusses a practitioner’s considerations with respect to understanding the nature of the internal audit function’s responsibilities, and the activities it performs, to determine whether to use the work of internal audit during the cybersecurity risk management examination. For situations in which the practitioner decides to use the work of the internal audit function in the cybersecurity risk management examination, chapter 2 also addresses the need to obtain written acknowledgment from management that internal auditors providing direct assistance will be allowed to follow the practitioner’s instructions without management’s interference, the evaluation of the objectivity and technical competence of members of the internal audit function, and the coordination of procedures with them, among other matters. This section discusses the practitioner’s responsibility to test the work of the internal audit function to determine whether it is adequate for the examination.

3.102 When using the work of the internal audit function, paragraph .40 of AT-C section 205 requires the practitioner to perform sufficient procedures, including reperformance, on the body of work of the internal audit function that the practitioner plans to use in order to evaluate whether such work is adequate for the practitioner’s purposes.

3.103 The nature, timing, and extent of procedures the practitioner performs in evaluating the adequacy of that work depends on the practitioner’s assessment of the significance of that work to the practitioner’s conclusions (for example, the significance of the risks that the controls are intended to mitigate). Such procedures usually consist of one or more of the following:

• Independent testing of items tested by the internal audit function (reperformance)

• Independent selection of items from the population tested by internal audit and the performance of testing of items of a similar nature that were performed by internal audit to independently evaluate internal audit’s conclusion

3.104 Some relevant factors in determining whether to use the work of the internal audit function to obtain evidence about the operating effectiveness of controls include the pervasiveness of the control, the potential for management override of the control, and the degree of judgment and subjectivity required to evaluate the effectiveness of the control. As the significance of these factors increases, so does the need for the practitioner, rather than the internal audit function, to perform the procedures, and conversely, as these factors decrease in significance, the need for the practitioner to perform the tests decreases.

3.105 To prevent undue use of the internal audit function in obtaining evidence, the practitioner uses less of the work of the internal audit function and performs more of the work directly when more judgment is involved in planning and performing relevant procedures or in evaluating the evidence obtained. Such situations are likely to occur when

• the assessed risk of material misstatement is higher;

• the internal audit function’s organizational status and relevant policies and procedures that adequately support the objectivity of the internal auditors are lower; and

• the level of competence of the internal audit function is lower.

3.106 The practitioner uses professional judgment in performing procedures to evaluate the work performed by the members of the entity’s internal audit function. As discussed in chapter 2, the practitioner is responsible for determining the work to be performed and obtaining sufficient appropriate evidence for the opinion. The practitioner has sole responsibility for the opinion expressed in the practitioner’s report, and that responsibility is not reduced by the practitioner’s use of the work of the internal audit function.

3.107 If the practitioner finds that the quality and extent of the work performed by the members of the entity’s internal audit function are not equivalent to the quality and extent of work the practitioner would have performed, the practitioner generally will perform additional procedures and consider the extent to which the work of the internal audit function may be used to obtain evidence.

3.108 In reviewing internal audit reports, the practitioner evaluates exceptions⁴ identified by the members of the entity’s internal audit function to determine whether those exceptions require the practitioner to alter the nature, timing, and extent of the practitioner’s procedures. The practitioner ordinarily corroborates exceptions identified by the members of the internal audit function and considers the extent of the exceptions, their nature and underlying causes, and whether additional procedures by the practitioner are necessary.

3.109 Another relevant factor in evaluating the adequacy of the work of the internal audit function is the adequacy of the sampling procedures used and whether the sampling procedures were appropriate and free from bias (that is, whether all items in the population have the same opportunity to be selected). The AICPA Audit Guide Audit Sampling provides additional guidance that may be useful to a practitioner who has decided to use audit sampling in performing procedures.

3.110 If the size of the sample used by the members of the entity’s internal audit function is less than the sample size the practitioner would have used, the practitioner generally would select additional items to achieve the required sample size. For example, if internal audit has selected a sample of 25 items for testing, the practitioner may determine that an additional 15 items need to be tested.

3.111 The responsibility to report on management’s description of the entity’s cybersecurity risk management program and the effectiveness of controls rests solely with the practitioner and cannot be shared with the internal audit function. Therefore, the judgments about the significance of deviations in the effectiveness of controls, the sufficiency of procedures performed, the evaluation of identified deficiencies, and other matters that affect the practitioner’s opinion are those of the practitioner. In making judgments about the extent of the effect of the work of the internal audit function on the practitioner’s procedures, the practitioner may determine, based on the risk associated with the controls and the significance of the judgments relating to them, that the practitioner will perform the work relating to some or all of the controls, rather than using the work performed by the internal audit function.

3.112 When using internal auditors to provide direct assistance to the practitioner, paragraph .42 of AT-C section 205 requires the practitioner to direct, supervise, and review the work of the internal auditors. The practitioner fulfills that responsibility by (a) informing the internal auditors of their responsibilities, the objectives of the procedures they are to perform, and matters that may affect the nature, timing, and extent of their procedures and by (b) supervising and reviewing the work performed by internal auditors in a manner similar to the review of work performed by the firm’s own staff.

3.113 Paragraph .44 of AT-C section 205 requires the practitioner, before the completion of the engagement, to evaluate whether the use of the work of the internal audit function or the use of internal auditors to provide direct assistance results in the practitioner still being sufficiently involved in the examination, given the practitioner’s sole responsibility for the opinion expressed.

Using the Work of a Practitioner’s Specialist

3.114 Chapter 2 discusses the practitioner’s responsibilities when a practitioner’s specialist will be used in the cybersecurity risk management engagement. Those responsibilities include (a) evaluating the specialist’s competence, capabilities, and objectivity; (b) obtaining an understanding of the specialist’s field of expertise to enable the practitioner to determine the nature, scope, and objectives of the specialist’s work and to evaluate the adequacy of that work; and (c) agreeing with the specialist on the terms of the engagement and other matters. In addition to those responsibilities, paragraph .36 of AT-C section 205 requires the practitioner to evaluate the adequacy of the work of the practitioner’s specialist for the practitioner’s purposes.

3.115 According to paragraph .36 of AT-C section 205, evaluating the adequacy of the work of the practitioner’s specialist involves consideration of the following:

1. The relevance and reasonableness of the findings and conclusions of the specialist and their consistency with other evidence

2. If the work of the practitioner’s specialist involves the use of significant assumptions and methods,

   1. obtaining an understanding of those assumptions and methods and

   2. evaluating the relevance and reasonableness of those assumptions and methods in the circumstances, giving consideration to the rationale and support provided by the practitioner’s specialist, and in relation to the practitioner’s other findings and conclusions

3. If the work of the practitioner’s specialist involves the use of source data that are significant to the work of the practitioner’s specialist, the relevance, completeness, and accuracy of that source data

3.116 If the practitioner determines that the work of the practitioner’s specialist is not adequate, paragraph .37 of AT-C section 205 requires the practitioner to

1. agree with the practitioner’s specialist on the nature and extent of further work to be performed by the practitioner’s specialist or

2. perform additional procedures considered appropriate in the circumstances.

Evaluating the Results of Procedures

3.117 The practitioner should evaluate the sufficiency and appropriateness of the evidence obtained and consider whether it is necessary to obtain further evidence to support his or her opinion on the description and the effectiveness of controls for the specified period of time.⁵ When making this evaluation, the practitioner should consider all relevant evidence, regardless of whether it appears to corroborate or to contradict the conclusion that the description is presented in accordance with the description criteria and the controls were effective to achieve the entity’s cybersecurity objectives based on the control criteria. Paragraphs .A49–.A53 of AT-C section 205 provide application guidance that might be helpful to the practitioner when making this evaluation.

3.118 The practitioner evaluates the results of all procedures performed and conducts both a quantitative (for example, rates of deviations in testing a control using a sample-based testing strategy) and qualitative analysis of whether identified description misstatements and deficiencies in the effectiveness of controls result in the description not being presented in accordance with the description criteria or in the controls not being effective to achieve one or more of the entity’s cybersecurity objectives. As an example, assume that, when investigating the follow-up and resolution of two identified security incidents, the practitioner determined that the resolution took longer than the management-prescribed resolution requirement to complete, but that difference was not material (for example, final resolution took two days longer than prescribed). In such an instance, the practitioner may conclude that the deficiencies were not material. However, if the practitioner’s testing determined that entity personnel failed to follow up at all for the two instances, he or she might conclude that the controls were not effective in achieving one or more criteria.

3.119 When evaluating the results of procedures, the practitioner investigates the nature and cause of any identified description misstatements and deficiencies or deviations in the effectiveness of controls and determines

• whether the identified description misstatements result in either the failure to meet one or more of the description criteria or in a presentation that could be misunderstood by users if the practitioner’s opinion were not modified to reflect the identified description misstatements.

• whether identified deviations are within the expected rate of deviation and are acceptable or whether they constitute a deficiency. If deviations are within the expected rate of deviation, the procedures that have been performed provide an appropriate basis for concluding that the control operated effectively throughout the specified period.

• whether identified deficiencies are likely to have, in the practitioner’s judgment, a pervasive effect on the achievement of the entity’s cybersecurity objectives (for example, whether more than one criterion would be affected).

• whether

— a previously tested control (or combination of controls) provides sufficient appropriate evidence about whether controls operated effectively or

— whether additional testing of the control or other controls is necessary to determine whether the controls were effective throughout the period to meet the control criterion. (If the practitioner is unable to apply additional procedures to the selected items, the practitioner should consider the reasons for this limitation and concludes on whether those selected items are deviations from the prescribed policy or result in a limitation of the scope of the engagement for the purpose of evaluating the sample. If the practitioner concludes that further evidence is needed, but the practitioner is unable to obtain it, paragraph .47 of AT-C section 205 states that the practitioner should consider the need to modify the opinion.)

• the magnitude of the effect of such deficiencies on the achievement of the entity’s cybersecurity objectives based on the control criteria.

• whether users could be misled if the practitioner’s opinion were not modified to reflect the identified deficiencies.

3.120 According to paragraph .A105 of AT-C section 205, the term pervasive describes “the effects on the subject matter of misstatements or the possible effects on the subject matter of misstatements, if any, that are undetected due to an inability to obtain sufficient appropriate evidence.” Based on that guidance, pervasive effects in the cybersecurity risk management examination might be those that are, in the practitioner’s professional judgment,

1. not confined to only specific aspects of the conclusion about control effectiveness or,

2. if so confined, represent or could represent a substantial proportion of the conclusion about control effectiveness.

3.121 Factors that may be considered when determining whether the identified deviations may have a pervasive effect on other controls include

• the effect that entity-level controls have on the operation of other controls. Deviations in entity-level controls often have a pervasive effect on other controls.

• the extent of the use of segmentation across the entity’s networks and systems. The greater the use of segmentation, the less likely it is that deviations in the operation of controls will have an effect on the operation of other controls.

• the extent to which deficiencies in certain key controls have a pervasive effect on other controls. For example, an entity that does not have effective controls over the detection of security events is unlikely to have an effective cybersecurity risk management program.

3.122 Paragraph .45 of AT-C section 205 also requires the practitioner to accumulate description misstatements or deficiencies identified during the engagement, other than those that are clearly trivial. In addition, the practitioner should accumulate deviations that have not been determined to rise to the level of a deficiency and consider whether, in the aggregate, they result in a deficiency.

3.123 If the practitioner identifies material description misstatements or material deficiencies in control effectiveness, the practitioner should modify the opinion. When modifying the opinion, the practitioner’s understanding of the nature and cause of the description misstatements and deficiencies enables the practitioner to determine how to appropriately modify the opinion. Chapter 4 of this guide discusses modifications of the practitioner’s report.

Responding to and Communicating Known or Suspected Fraud, Noncompliance With Laws or Regulations, Uncorrected Misstatements, or Internal Control Deficiencies

Known or Suspected Fraud or Noncompliance With Laws or Regulations

3.124 As discussed in chapter 2, the practitioner has a responsibility to consider known or suspected incidents of fraud and noncompliance with laws or regulations. The practitioner determines the effect of such incidents on management’s description of the entity’s cybersecurity risk management program, on the effectiveness of controls to achieve the entity’s cybersecurity objectives based on the control criteria, and on the practitioner’s report. Additionally, the practitioner communicates such information to appropriate parties.

3.125 When incidents of fraud or suspected fraud are identified during the engagement, the practitioner is expected to respond appropriately. For example, unless prohibited by law, regulation, or ethics standards, appropriate responses may include the following:

• Discussing the matter with senior management (and the engaging party, if different) and other appropriate party(ies), unless senior management is suspected to have committed the fraud. If the practitioner suspects fraud involving senior management, the practitioner should communicate these suspicions to those charged with governance and discuss with them the nature, timing, and extent of procedures necessary to complete the examination.

• Requesting that senior management (and the engaging party, if different) consult with an appropriately qualified third party, such as the entity’s legal counsel or a regulator

• Considering the implications of the matter in relation to other aspects of the engagement, including the practitioner’s risk assessment and the reliability of written representations from management (and the engaging party, if different)

• Obtaining legal advice about the consequences of different courses of action

• Communicating with third parties (such as a regulator)

• Withdrawing from the engagement

3.126 The actions noted in the preceding paragraph may also be appropriate in response to noncompliance or suspected noncompliance with laws or regulations identified during the engagement. In addition, the practitioner may decide to describe the matter in a separate paragraph in the practitioner’s report, unless the practitioner

1. is precluded by management (or the engaging party, if different) from obtaining sufficient appropriate evidence to evaluate whether noncompliance that may be material to the conclusion about the effectiveness of controls to achieve the entity’s cybersecurity objectives has, or is likely to have, occurred. In this situation, there is a scope limitation which precludes the practitioner from expressing an opinion on the effectiveness of controls to achieve the entity’s cybersecurity objectives; accordingly, the practitioner would disclaim an opinion.

2. concludes that the noncompliance results in the entity’s failure to achieve the entity’s cybersecurity objectives based on the control criteria. In this situation, the practitioner expresses a modified opinion.

Communicating Incidents of Known or Suspected Fraud, Noncompliance With Laws or Regulations, Uncorrected Misstatements, or Internal Control Deficiencies

3.127 In addition to responding to known and suspected fraud and non-compliance with laws or regulations, the practitioner should communicate information regarding those matters, along with information regarding any uncorrected description misstatements or material deficiencies, to the appropriate levels of management (and to the engaging party, if different). The practitioner may also consider whether to communicate other matters.

3.128 If the practitioner identifies or suspects noncompliance with laws or regulations that are not relevant to the subject matters of the cybersecurity risk management examination, the practitioner should determine whether he or she has a responsibility to report the identified or suspected noncompliance to parties other than management (and the engaging party, if different).

3.129 The practitioner may be precluded from reporting such incidents to parties outside the entity because of the practitioner’s professional duty to maintain the confidentiality of client information. However, the practitioner’s legal responsibilities may vary by jurisdiction and, in certain circumstances, the duty of confidentiality may be overridden by statute, law, or courts of law. A duty to notify parties outside the entity may exist

• in response to a court order or

• in compliance with requirements for examinations of entities that receive financial assistance from a government agency.

3.130 Because potential conflicts with the practitioner’s ethical and legal confidentiality obligations may be complex, the practitioner may decide to consult with legal counsel before discussing noncompliance with parties outside the entity.

Obtaining Written Representations From Management

3.131 During the cybersecurity risk management examination, management makes many oral and written representations to the practitioner in response to specific inquiries or through the presentation of the description of the entity’s cybersecurity risk management program and management’s assertion. Such representations from management are part of the evidence the practitioner obtains. However, they cannot replace other evidence the practitioner could reasonably expect to be available, nor do they provide sufficient appropriate evidence on their own about any of the matters with which they deal. Furthermore, the fact that the practitioner has received reliable written representations does not affect the nature or extent of other evidence that the practitioner obtains.

3.132 Written representations from management ordinarily confirm representations explicitly or implicitly given to the practitioner, indicate and document the continuing appropriateness of such representations, and reduce the possibility of a misunderstanding concerning the matters that are the subject of the representations.

3.133 Paragraph .50 of AT-C section 205 indicates that, in an examination engagement, a practitioner should request written representations in the form of a letter from the responsible party. The representations in the cybersecurity risk management examination should

1. include management’s assertion about the subject matters⁶ based on the criteria.⁷

2. state that

   1. all relevant matters are reflected in the measurement or evaluation of the subject matters or assertion,

   2. all known matters contradicting the subject matters or assertion and any communication from regulatory agencies or others affecting the subject matters or assertion have been disclosed to the practitioner, including communications received between the end of the period addressed in the written assertion and the date of the practitioner’s report.

3. acknowledge responsibility for

   1. the subject matters and the assertion,

   2. selecting the criteria, and

   3. determining that such criteria are appropriate for management’s purposes.

4. state that any known events subsequent to the period (or point in time) of the subject matters being reported on that would have a material effect on the subject matters or assertion have been disclosed to the practitioner.

5. state that management has provided the practitioner with all relevant information and access.

6. state that management believes the effects of uncorrected misstatements (description misstatements and deficiencies) are immaterial, individually and in the aggregate, to the subject matters.

7. state that management has disclosed to the practitioner

   1. all deficiencies in internal control relevant to the cybersecurity risk management examination of which it is aware;

   2. its knowledge of any actual, suspected, or alleged fraud or noncompliance with laws or regulations affecting the subject matters;

   3. identified security incidents that affected the entity’s achievement of its cybersecurity objectives; and

   4. other matters the practitioner deems appropriate (for instance, discussion of matters considered material).

3.134 When written representations are directly related to matters that are material to the subject matter, the practitioner should

1. evaluate their reasonableness and consistency with other evidence obtained, including other representations (oral or written) made by management, and

2. consider whether those making the representations can be expected to be well informed on the particular matters.

3.135 The written representations required are separate from, and in addition to, management’s written assertions. They are usually made in the form of a representation letter addressed to the practitioner, dated as of the date of the practitioner’s report, and they should address the subject matters and periods referred to in the practitioner’s opinion.

Requested Written Representations Not Provided or Not Reliable

3.136 Paragraph .55 of AT-C section 205 provides guidance to the practitioner when

• management has not provided one or more of the requested representations;

• the practitioner concludes that there is sufficient doubt about the competence, integrity, ethical values, or diligence of those providing the written representations; or

• the practitioner concludes that the written representations are otherwise not reliable.

3.137 In such circumstances, the guidance in that paragraph states that the practitioner should

• discuss the matter with the appropriate party(ies);

• reevaluate the integrity of those from whom the representations were requested or received and evaluate the effect that this may have on the reliability of representations and evidence in general; and

• if any of the matters are not resolved to the practitioner’s satisfaction, take appropriate action.

3.138 Ordinarily, in the cybersecurity risk management examination, management’s refusal to furnish evidence in the form of written representations constitutes a limitation on the scope of the examination sufficient to preclude an unmodified opinion on either the description or the effectiveness of controls. Usually, the scope limitation is sufficient to cause the practitioner to disclaim an opinion on both or to withdraw from the engagement.

Subsequent Events and Subsequently Discovered Facts

3.139 Events or transactions may occur after the specified period of time covered by the examination engagement, but prior to the date of the practitioner’s report, that could have a significant effect on the description of the entity’s cybersecurity risk management program or the effectiveness of controls within that program. In such circumstances, disclosure in the description or in management’s assertion may be necessary to prevent users of the cybersecurity risk management examination report from being misled.

3.140 The following are examples of events that could affect the description of the entity’s cybersecurity risk management program or management’s assertion:

• After the period covered by the examination engagement, management discovered that, during the last quarter of that period, the IT security director provided all the programmers with access to the production data files, enabling them to modify data.

• After the period covered by the examination engagement, management discovered that a confidentiality breach occurred at the entity during the period covered by the practitioner’s report.

3.141 Paragraph .48 of AT-C section 205 requires the practitioner to inquire of management (and if different, the engaging party) about whether it is aware of any such events. If such events exist, the practitioner should apply appropriate procedures to obtain evidence regarding the events. For example, the practitioner may obtain evidence by inquiring about and considering information regarding the effectiveness of controls within the entity’s cybersecurity risk management program by inspecting

• relevant internal auditors’ reports issued during the subsequent period.

• other practitioners’ reports issued during the subsequent period.

• relevant regulatory agencies’ reports issued during the subsequent period.

• reports on other professional engagements for that entity.

3.142 Paragraph .48 of AT-C section 205 does not require the practitioner to perform any procedures regarding the description of the entity’s cybersecurity risk management program, the effectiveness of controls within that program, or management’s assertion, after the date of the practitioner’s report. However, paragraph .49 of AT-C section 205 clarifies that the practitioner is responsible for responding appropriately to facts that become known after the date of the report that, had they been known as of the report date, may have caused the practitioner to revise the report.

3.143 After obtaining information about an event, the practitioner determines whether the facts existed at the date of the report and, if so, whether persons who would attach importance to these facts are currently using, or likely to use, the cybersecurity risk management examination report (which includes management’s description and assertion and the practitioner’s report). The practitioner may do this through discussions with management and other appropriate parties and through the performance of additional procedures that the practitioner considers necessary to determine whether the description, assertion, and practitioner’s report need revision or whether the previously issued report continues to be appropriate.

3.144 Specific actions to be taken at that point depend on a number of factors, including the time elapsed since the date of the practitioner’s report and whether issuance of a subsequent report is imminent. Depending on the circumstances, the practitioner may determine that notification of persons currently using or likely to use the practitioner’s report is necessary. This may be the case, for example, when

• the cybersecurity risk management examination report is not to be relied upon because

— the description, management’s assertion, or the practitioner’s report needs revision or

— the practitioner is unable to determine whether revision is necessary and

• issuance of a subsequent practitioner’s report is not imminent.

3.145 If the practitioner believes the event is of such a nature and significance that its disclosure is necessary to prevent users of the cybersecurity risk management examination report from being misled, the practitioner should determine whether information about the event is adequately disclosed in the description or management’s assertion. For example, assume that, after the period covered by the examination but prior to the date of the practitioner’s report, management learns of a security incident involving the loss of customers’ personal information. After investigation, management determines that the incident stemmed from an otherwise unknown vulnerability in its system; furthermore, that vulnerability existed during the examination period. In this example, the practitioner ordinarily would conclude that the matter should be disclosed in the description and assertion. If it is not, the practitioner’s course of action depends on the practitioner’s legal and ethical rights and obligations. Therefore, the practitioner may consider seeking legal advice before deciding on a course of action. Appropriate actions may include

1. disclosing the event (including a description of the nature of the event and its effect on the description, assertion, or report) in the practitioner’s report and modifying the related practitioner’s opinion, and

2. withdrawing from the engagement.

Subsequent Events Unlikely to Have an Effect on the Practitioner’s Opinion

3.146 The practitioner may have determined that the event discovered subsequent to the period covered by the examination engagement would likely have had no effect on either the presentation of the description in accordance with description criteria or the effectiveness of controls because the underlying situation did not exist until after the period covered by the cybersecurity risk management examination report. However, the matter may be sufficiently important to warrant disclosure by management in its description and, potentially, emphasis by the practitioner in the practitioner’s report. The following are examples of such events:

• The entity was acquired by another entity.

• The entity experienced a significant operating disruption.

• A data center-hosting entity that provides applications and technology that enable user entities to perform essential business functions made significant changes to its information systems, including a system conversion or significant outsourcing of operations.

Documentation

3.147 Paragraphs .34–.41 of AT-C section 105 provide requirements regarding the documentation that should be prepared for an attestation engagement. Those paragraphs address matters such as the timeliness of the documentation, how to make necessary changes to the documentation after the original preparation date, retention of engagement documentation, confidentiality of documentation, and the need to document situations in which the practitioner judges it necessary to depart from a relevant presumptively mandatory requirement.

3.148 Additionally, paragraphs .87–.89 of AT-C section 205 discuss the practitioner’s responsibilities for preparing and maintaining documentation that is appropriate to an examination engagement. The practitioner’s documentation in a cybersecurity risk management examination is the principal record of attestation procedures applied, information obtained, and conclusions or findings reached by the practitioner. The quantity, type, and content of documentation are matters of the practitioner’s professional judgment. However, the documentation should be sufficient to determine

1. the nature, timing, and extent of the procedures performed to comply with AT-C sections 105 and 205 and applicable legal and regulatory requirements, including

   1. the identifying characteristics of the specific items or matters tested;

   2. who performed the engagement work and the date such work was completed;

   3. the discussions with management or others about findings or issues that, in the practitioner’s professional judgment, are significant, including the nature of the significant findings or issues discussed, and when and with whom the discussions took place;

   4. when management will not provide one or more of the requested written representations or the practitioner concludes that there is sufficient doubt about the competence, integrity, ethical values, or diligence of those providing the written representations or that the written representations are otherwise not reliable, the matters in paragraph.55 of AT-C section 205 (see paragraphs 3.136–.137 of this guide); and

   5. who reviewed the engagement work performed and the date and extent of such review.

2. the results of the procedures performed and the evidence obtained.

3.149 In addition to the items in the preceding paragraphs, documentation in the cybersecurity risk management examination should include the following:

• If the practitioner has identified information that is inconsistent with the practitioner’s final conclusions, how the practitioner addressed the inconsistency

• If, after the date of the report, the practitioner becomes aware of facts that may have caused the practitioner to revise the report had they been known at the time of the report,

— the circumstances encountered;

— any new or additional procedures performed, evidence obtained, and conclusions reached and their effect on the report; and

— when and by whom the resulting changes to the documentation were made and reviewed

3.150 As in other attestation engagements, documentation in the cybersecurity risk management examination would ordinarily also include a record of

• issues identified with respect to compliance with relevant ethical requirements and how they were resolved.

• conclusions on compliance with independence requirements that apply to the engagement and any relevant discussions with the firm that support these conclusions.

• conclusions reached regarding the acceptance and continuance of client relationships and attestation engagements.

• the nature and scope of, and conclusions resulting from, consultations undertaken during the course of the engagement.

• if the practitioner uses the work of the internal audit function, other practitioners, or specialists, documentation of conclusions reached by the practitioner regarding the evaluation of the adequacy of the work and the procedures performed on that work.

3.151 Paragraphs .A117–.A119 of AT-C section 205 provide additional application guidance that might be helpful to a practitioner when deciding what to document in the cybersecurity risk management examination.

Management’s Responsibilities at or Near Engagement Completion

3.152 Management’s responsibilities at or near completion of the cybersecurity risk management examination include

• modifying the description, if appropriate (chapter 4 describes a number of situations in which the practitioner would recommend that management modify the description);

• modifying management’s written assertion, if appropriate;

• providing written representations (as discussed beginning in paragraph 3.131);

• informing the practitioner of subsequent events; and

• distributing the report to appropriate parties.

Modifying Management’s Assertion

3.153 As discussed in chapter 2, management provides the practitioner with a written assertion about whether the description is presented in accordance with the description criteria and whether the controls within the program were effective to achieve the entity’s cybersecurity objectives. Manage-ment’s written assertion is generally expected to align with the practitioner’s opinion by reflecting the same modifications.

3.154 The following is an example of modifications (indicated with bold text) that might be made to management’s assertion when the description is not presented in accordance with the description criteria and the practitioner has modified the opinion in his or her report:

[Assertion paragraph]

Except for the matter described in the following paragraph, we assert that the description throughout the period [date] to [date] is presented in accordance with the description criteria. We have performed an evaluation of the effectiveness of the controls within the cybersecurity risk management program throughout the period [date] to [date] using the [name of the control criteria, e.g., the criteria for security, availability, and confidentiality set forth in TSP section 100, 2017 Trust Services Criteria for Security, Availability, Processing Integrity, Confidentiality, and Privacy (AICPA, Trust Services Criteria) or other suitable criteria] (control criteria). Based on this evaluation, we assert that the controls were effective to achieve the entity’s cybersecurity objectives throughout the period [date] to [date] based on the control criteria.

The description of our cybersecurity risk management program states that the entity has physical access controls that incorporate biometric devices and individual PINs. Although such controls have been implemented throughout ABC’s main facility, they have not been consistently implemented in our other three facilities.

3.155 The following is an example of modifications (indicated with bold text) that might be made to management’s assertion when controls were not effective to achieve the entity’s cybersecurity objectives and the practitioner has modified that component in his or her report:

[Assertion paragraph]

We assert that the description throughout the period [date] to [date] is presented in accordance with the description criteria. We have performed an evaluation of the effectiveness of the controls within the cybersecurity risk management program throughout the period [date] to [date] using the [name of the control criteria, e.g., the criteria for security, availability, and confidentiality set forth in TSP section 100, 2017 Trust Services Criteria for Security, Availability, Processing

Integrity, Confidentiality, and Privacy (AICPA, Trust Services Criteria) or other suitable criteria] (control criteria). Based on this evaluation, we assert that, with the exception of the matter described in the following paragraph, the controls were effective to achieve the entity’s cybersecurity objectives throughout the period [date] to [date] based on the control criteria.

The description of our cybersecurity risk management program states on page 8 that application changes are tested prior to their implementation. The procedures, however, do not include a requirement for scanning application code for known vulnerabilities prior to placing the change into operation. As a result, the controls were not effective to meet criterion CC8.1, The entity authorizes, designs, develops or acquires, configures, documents, tests, approves, and implements changes to infrastructure, data, software, and procedures to meet its objectives.

3.156 If management is unwilling to modify its assertion to align with the practitioner’s opinion, the practitioner should consider the implications on the practitioner’s report. For example, the practitioner should consider whether report users are likely to misunderstand a cybersecurity risk management examination report that includes management’s assertion and the practitioner’s report, when management and the practitioner have reached and expressed different conclusions with respect to either the description or the effectiveness of controls in the same document. If the practitioner believes it is likely that such a report will be misunderstood by users, the practitioner may decide to withdraw from the engagement.