A corollary to assume breach is to assume control failure. In the words of many a CIO, if you don’t check it then it wasn’t done. Anyone who has managed operations or service vendors knows that some IT workers have a different definition of done than you. Given time and exposure to the real world, things drift from their modeled description. Policies don’t match what people are doing. Project status updates are inflated. Network diagrams aren’t current or complete. Log data isn’t captured or if it is, the data slumbers unanalyzed somewhere. People leave the organization but their accounts remain active. Implementation projects get paused mid-implementation because of operational emergencies, but they are never resumed. Maintenance slips and patching doesn’t complete. I’m not pessimistic—this just happens in a large, busy IT organization. However, if a control isn’t working as described by policy, then you need to find it and fix before the auditors or attackers spot them. That is what internal audit is about.
The Role of Internal Audit
Internal audit is a role in your organization with the same kind of duties as the external auditor but they are on staff and embedded in the security program permanently. In larger organizations, internal audit is handled by its own distinct department and it often covers multiple audit compliance systems. If you are pursuing an ISO 27001 audit , then the internal audit role is a mandatory function. For other audit requirements, it is not necessarily required but it is still a useful and powerful role to have. There are two major requirements that internal auditors must meet: independence and competence.
Internal Auditor Independence
Since internal audit is about questioning the design and effectiveness of the entire security program, the internal auditor must be free to express an honest opinion without fear of backlash. Any conflict of interest between the security team, the IT operational team, and the internal auditor will produce a chilling effect on the quality of reporting. If the auditor finds a mismatch between goal and reality, it must be disclosed completely and candidly to management. This means the segregation of duties concept is required for this role .
The internal audit must not report to IT or even the security team. This auditor should report to the management, which is to say, the same executive (or higher) that the chief security officer role reports to. If the security team is under the chief operating officer, then internal audit should be a direct report as well. I have also seen internal audit report to the CEO or an entirely different wing of the organization. For example, internal audit could be attached to the financial department, which may already have an internal audit function for accounting. Figure 22-1 shows a sample organization chart illustrating this.
Figure 22-1. Sample org chart with internal audit
There have been cases where an internal auditor attached to the IT department was fired because they uncovered fraud. Unfortunately, this lack of independence within internal audit was not uncovered until the subsequent criminal investigation after much damage was done .
Internal Auditor Competence
Just as important as auditor independence is that the auditor understands the technology requirements and the compliance requirements to be met. In the best case of an untrained internal audit team , they may miss control failures and potential threats. In the worst case, the auditor could be misled or steered away from potential problems by those being audited. If you don’t know what exactly to look for regarding technology and security, it’s easy to be confused or mistaken about how something functions. The lapse will be worse if you have a malicious insider actively working to cover up a mistake or crime. You also don’t want a situation where internal audit has consistently given a security control a passing grade only to have an external auditor find significant fault in it .
To assist with this problem, there are internal auditor training classes. The ISO training institutions offer multi-day ISO 27001 internal audit training,which can be found by running that phrase through your favorite search engine . Some organizations send their PCI DSS internal auditors to full QSA training to ensure that they are fully qualified in the standard. In the financial accounting world, there is the Institute for Internal Auditors,1 which offers training and certification in the general practice of the internal process auditing. However, those auditors should also undergo additional training to ensure that they understand the technology and specific compliance requirements that they will audit against. In most cases, internal auditors are not formally beholden to a certifying standards body, so their training and certification have a different weight than a true independent auditor.
Beyond the specific technical skills, auditors also need to be competent in acquiring, interpreting, analyzing, and reporting the data. They need to have an attitude of professional skepticism and question everything they are told. Even machine-produced results should be examined to ensure that they are telling the whole story. A useful skill for internal auditors is how to test for internal fraud. There is training and certification in this area from the Association for Certified Fraud Examiners. 2 An auditor with technical, compliance, and fraud skills is a triply valuable individual to have on staff.
How Small Can the Role Go?
Larger organizations can afford the resources to dedicate employees to internal audit but smaller ones may not. This doesn’t mean you give up on this vital function. Internal audit is a role, not necessarily a job title. All the audit needs is independence and capability. I’ve seen smaller organizations create part-time internal auditors out of external consultants, internal financial auditors (with additional training), and benched personnel between projects. If you aren’t looking to hit ISO 27001 certification, you can even push the limit of segregation of duties and have members of the security team do internal audit duties as well. Internal non-dedicated auditors can still provide the benefit from redundancy and cross-checks for other team members .
To Heal, Not to Punish
Internal audit exists to make the organization’s security program stronger. I have seen some organizations where the segregation of duties has gone too far. A healthy rivalry between internal audit and IT security is tolerable and in some cases, worth encouraging. However, you should avoid cultures where internal audit is out to get the IT team. Internal auditors should never set out to trap or trick people into failing audit. Even if this isn’t the intention, this perception can be damaging to morale and create performance impacts. This only fosters an adversarial relationship where both parties end up deceiving and withholding information from each other, and ultimately impedes the organization’s security program.
The goal is to have everyone working together toward an evolving security program with the best controls for current risks that the organization can afford. Consider the stance between a culture of negative error and a culture of safety. Negative error cultures are defensive where employees work to avoid getting in trouble or being blamed for mistakes. Negative error cultures focus on exclusively managing to the letter of the law and nothing beyond. They stifle speaking up about problems as mistakes are seen as personal failings and not systemic issues. Defensive decision-making flourishes, where choices are made based on minimizing culpability.
A culture of safety is about prevention, transparency, and continuous improvement. In a safety culture, everyone functions in some way like an internal auditor, looking out for mistakes and deficiencies in order to promote the greater goal of safety. Problems and findings are not blamestormed, but analyzed to uncover the organizational processes and design issues that led to them. In a culture of safety, reminders and reviews from internal auditors are valued and heeded, rather than feared and avoided.
Check Before the Auditors Check
The internal auditor’s job is to fix things before there are severe impacts or external audit findings. They should walk through the entire stack of controls to ensure that everything functions as described in policy. This means getting up to their elbows in firewall rules, user accounts, change control tickets, and security policy decisions.
Usually, lots of cruft, or leftover obsolete and unnecessary things, are found during these checks, such as the following:
Confidential data stored and forgotten on systems that should have been removed or encrypted
Systems that should have been decommissioned but still left online
Expired licenses and subscriptions for operational and security systems
Live user accounts for individuals who had left the organization
Zombie service accounts still active but no longer needed
Excessive management overrides of internal controls for policies that don’t fit
Unrecorded (and therefore unauthorized) changes to key systems under change control
Half-installed controls from rushed projects or distracted implementation teams
Missing patches, hardening, or control installs
Missing root cause analyses from incidents
Firewall rules that are no longer needed, but have not been decommissioned
All of these things are the typical meat and potatoes of an internal auditor’s review. Any one of these things can lead to a security incident or audit finding .
The Internal Audit Process
Just like an external audit, the internal audit process should involve both a document review and personnel interviews. Overall, the internal audit plan should look a lot like an external audit plan, but spread out over a longer period. External auditors are usually onsite a few weeks at most per year, internal auditors should take full advantage of the fact that they live onsite. A schedule should be set up for the internal auditor to sample each control and process area over the audit period (usually a year), circling back to key controls on a regular basis. The key controls should be checked at least quarterly are user management, authorized changes, access control less and data backup functionality. Whatever the audit schedule is, it should not be published outside of internal audit too far in advance. To the audited parts of the organization, the audits should be somewhat unpredictable.
Like external auditors, the internal audit plan involves a pull list of key records to examine in each control area. Executive management should ensure full cooperation with internal auditors. Refusing to comply in a timely and transparent manner with internal audit should be considered an audit finding itself.
Measuring a Control
Control measurement isn’t just limited to internal audit, as the security team will also be involved in tracking the progress of their program. Because of their role, internal auditors are in a good position to take and track control metrics. Over the period, internal auditors verify and inspect every process and control in the organization related to security.
This measuring process necessitates that the controls and processes be designed so that they produce observable artifacts on the functionality. Firewalls should be capable of creating logs and configuration reports. Meetings should have minutes and agendas. User access requests should have paper trails and authorization tickets. The auditor saying applies here as well: if it isn’t documented, it does not exist. The failure to produce verifiable proof from an active control or process is an audit finding .
However, it still may fall upon the internal auditor to devise their own tests and checks for controls and processes. The goal is to find a measure that yields good information on the intent of the thing being measured. If a control is implemented based on a policy to reduce a particular type of risk, then there should be a way to measure the implementation against policy and test its risk-reduction capability. Sometimes this is easy and sometimes it is a challenge. Let’s look at some ideas for doing this in Table 22-1.
Table 22-1. Controls and How to Measure Them
Control or Process | Possible Check |
|---|---|
Asset inventory | Last time done. Random sample spot-check of live assets against inventory list. |
Risk analysis | Last time done. Review of risk assumptions. Review of compliance analysis. Review of threat analysis. Review of impact analysis. |
Scope | Review of scope perimeter. Review of scope perimeter controls. Check scoped assets. Check operational processes. Look for scope data leakage. Review business processes for impacts to scope. |
Security training | Review training attendance records. Last update of training material. Training topic coverage and relevance to risks and compliance. Review training test results. Review past security incidents related to training topics. |
Security Policy | Review user policy sign-off records. Last policy update. Policy coverage and relevance to risks and compliance. |
Standards and Policies | Coverage of documents to requirements. Completeness of documents. Last updated. |
Security department | Review of security team job descriptions. Last security training attended. Certifications completed . |
IT department | Review of IT job descriptions. Last IT operational training attended. Review segregation of duties. |
General controls | Review key controls for defense in depth. Check control deployment. Review control operational records. Review Control maintenance records. |
Change control | Look for unauthorized changes done. Review unauthorized changes detected. Scan change control records for documentation completeness. |
Authentication | Reconcile UserIDs to current HR records. Check authentication implemented vs standard across systems. Check password settings to standard. Check two-factor inventory. |
Authorization | Review authorization settings per UserID and groups looking for excessive or obsolete rights. |
Firewalls | Review firewall configurations. Review firewall rule documentation records. Check for software updates. Review maintenance records. Check the number of administrators on the firewall. Review firewall change logs. Check firewall configuration vs published standards. |
Intrusion Detection/Prevention | Review signature update dates. Review maintenance records. Review records of alert responses and analysis. Check coverage of system on critical net flows. Check log retention. |
Network devices | Review VLAN configurations on critical segregations. Check configured administrators per device. Review change-logging records. Check actual configuration vs. published standard . |
Vulnerability scans | Check that it was done at least quarterly. Review depth and breadth of scan. Verify competence of scanner operator. Review scan settings to ensure updated. Ensure that results were reviewed. Ensure that remediation was done. |
Penetration tests | Check that it was done at least annually. Review depth and breadth of test. Verify that competence of tester. Ensure that results were reviewed. Ensure that remediation was done . |
Internet-visible applications | Check that application security was reviewed. Review last security scan done. Check actual hardening vs. published standards. Ensure that DMZ and network segregation are in use. Check last code update . |
Network encryption | Check actual configuration vs. published standard. Review last update of standards. Review last vulnerability scan. Review configuration documentation. Check software updates. Review maintenance records. Check admins per device. Review change logging. Ensure that encryption has not gone obsolete. |
Storage encryption | Check actual configuration vs. published standard. Check coverage of encryption across devices. Check last update of standards. Review configuration documentation. Check change logging. |
Crypto key management | Actual configuration vs. published standard. Key management records. Admins per system. Key inventory. Ensure that encryption has not gone obsolete. |
Physical perimeter security | Check key/card key inventory vs. current employees. Ensure that PIN codes are being properly rotated on schedule. Verify proper locks function on doors (closets. cages. rooms). Check automatic door closing mechanisms. Ensure that co-location facility list is current. |
Visitor security | Review visitor log record. Spot-check the usage of badges. Review retention of logs. Spot-check for unescorted visitors. |
Cameras | Check coverage and visibility of cameras. Ensure that cameras are properly recording. Review retention of video log to standard. |
Physical security | Check for unsecured confidential data sitting out unprotected. Check media inventory vs. actual media. Ensure that assets are locked up. Check that media is properly labeled. Review disposal records. Ensure that shred bins are being used. Spot-check recycle bins for confidential documents not shredded . |
Business continuity | Ensure that continuity tests are being done regularly. Check last update on plan. Check coverage of plan vs. possible outages. Review plan for training records. Check plan relevance to current environment. Check root cause analysis reports vs. known outages. Review call trees and notification mechanisms. |
Incident response | Review record of past incidents. Review last update on plan. Check plan training. Check plan relevance to current environment. Review post-mortems to known incidents. Review call trees and notification mechanisms . |
Third-party security | Check security reviews vs. actual third parties. Check when last completed for each. Review depth and breadth of review. Check that connections are documented. Review third-party perimeter standards vs. actual implementation. |
Findings and non-conformities should mean a frequent and often revisit of the activity until assurance is given that no additional findings will turn up. Even then, controls that had findings should be given higher priority thereafter in future internal audits.
ISO Standard for Measurement
For more ideas on how to measure control effectiveness, consider looking at ISO 27004. This standard provides details on how to measure the effectiveness of an information security management system. If you are implementing and auditing against ISO 27001, I strongly suggest that you investigate this standard. Even if you’re not, the standard includes detailed instructions on how to build indicators and metrics for IT security controls. You can find out more about it at http://www.iso.org/iso/catalogue_detail?csnumber=42106 .
Publish to Management
As part of the internal audit cycle, auditors need to provide reports to management. An audit is useless if there are no adjustments based on the findings. Management is responsible for reading and acting upon the results in order to improve the security program. In ISO 27001, this is a major requirement of the ISMS process. This also means that internal auditors need to report their findings in a timely and understandable manner. The internal audit report should look like an informal, shorter version of an external audit report. It should have the following elements :
Executive summary
Specific compliance requirements being tested (PCI DSS v3.1 Requirement 1 – Firewall, etc.)
Methodology and time/date of testing
Audit tests performed and results
Participants (name, job, role)
Specific controls tested and results
Recommended corrective actions (if any)
Regarding executive summaries, they should be written with an executive in mind. Remember the chapter on talking to the suits and keep the writing clear, factual, and to the point. Don’t fluff up the summary with the specifics of the internal audit process, those can be reported later in the main report. The summary should just contain what was tested, the findings uncovered, and an opinion on how an external audit of this area would play out. The goal is that summary is actionable, so that a decision can be made.
Keep Records
Since the internal audit function can be subject to external audit, there should be proper documentation and record keeping. These records should include the following:
Description of internal audit team and roles
Schedule of audits (past and present)
Internal audit methodology (it should align with organizational priorities, risks, and asset values)
Audit reports to management including details on findings
Management’s documented response for corrective actions
Internal audit’s follow up on corrective actions
Since these audits may contain confidential information regarding controls and organizational processes, the reports should be classified as confidential and access controlled .