Key concepts you will need to understand:
✓ Methods and approaches for designing and improving business procedures (e-business, B2B, BPR)
✓ Business process controls (management, automated, and manual controls)
✓ Business performance indicators (balanced scorecard, key performance indicators [KPI])
✓ Business project organization, management, and control practices
✓ Project progress monitoring and reporting mechanisms
✓ Project success criteria and pitfalls
✓ Corporate governance risk and control frameworks
Techniques you will need to master:
✓ Evaluate the efficiency and effectiveness of information systems in supporting business processes, through techniques such as benchmarking, best practice analysis, or business process re-engineering (BPR), to ensure optimization of business results
✓ Evaluate the design and implementation of programmed and manual controls to ensure that identified risks to business processes are at an acceptable level
✓ Evaluate business process change projects to ensure that they are properly organized, staffed, managed, and controlled
✓ Evaluate the organization’s implementation of risk management and governance
The evaluation of the efficiency and effectiveness of an organization’s IT program involves reviewing the IT governance structure as well as its alignment with the organization’s strategy. The IT organization must also manage the risks associated with ongoing development and operations. The IT organization should have a risk-management program that utilizes internal controls and best practices to mitigate risks to an acceptable level. As a part of risk management, the IT organization should have formal documented methodologies for managing business process change to include organization, management, controls, and measurement. The IS auditor should ensure that IT is aligned with corporate goals and that the benefit of IT is maximized while risk is minimized.
The standard approach to improving business processes is to identify specific areas to be reviewed, document the existing baseline process(s), and identify areas for improvement. After improvement areas have been identified, they should be presented to senior management for prioritization and implementation. Upon implementation of the business processes, the organization should monitor the new processes against the baseline and establish a continuous improvement process. Known as business process re-engineering (BPR), this usually successfully reduces manual interventions and controls within the organization.
ISACA defines benchmarking as the continuous, systematic process of evaluating the products, services, and work processes of organizations, recognized as representing best practices for the purpose of organizational improvement. The purpose of identifying a benchmarking partner is to find a work process in your industry that is identified as having the qualities that your organization would like to re-engineer to (success, quality, excellence, and so on). ISACA outlines the following steps in a benchmarking exercise:
Plan. In the planning stage, critical processes are identified for the benchmarking exercise. The benchmarking team should identify the critical processes and understand how they are measured, what kind of data is needed, and how that data needs to be collected.
Research. The team should collect baseline data about its own processes before collecting this data about others. The next step is to identify the benchmarking partners through sources such as business newspapers and magazines, quality award winners, and trade journals.
Observe. The next step is to collect data and visit the benchmarking partner. There should be an agreement with the partner organization, a data-collection plan, and a method to facilitate proper observation.
Analyze. This step involves summarizing and interpreting the data collected, analyzing the gaps between an organization’s process and its partner’s process, and converting key findings into new operational goals.
Adapt. Adapting the results of benchmarking can be the most difficult step. In this step, the team needs to translate the findings into a few core principles and work down from the principles to strategies and action plans.
Improve. Continuous improvement is the key focus in a benchmarking exercise. Benchmarking links each process in an organization with an improvement strategy and organizational goals.
The IS auditor must ensure that the change efforts are consistent with the culture and strategic plan of the organization, and that the change efforts reduce negative impact on the organization’s staff. In addition, the auditor must ensure that key controls, if required, are engineered into the new process. If key controls are removed as a part of the re-engineering effort, the IS auditor must ensure that all risks associated with these controls are communicated to and accepted by management.
In today’s competitive landscape, the continuous improvement of business processes no longer ensures an organization’s survival. Business change is primarily driven by customer needs for new and improved products and services. If an organization cannot provide these products and services, customers have the option of turning to other organizations that can provide these products and services. Business process re-engineering (BPR) provides an accelerated means of process improvement by assuming that existing business processes do not work; therefore, the re-engineering effort can focus on a new processes by defining a future state (to be).
After the future state has been defined, the re-engineering team can create an action plan based on the gap between current processes and the future state. The re-engineering team and management then can create the transition plan and begin to implement the changes. To help ensure the success of the re-engineering effort, determining the scope of areas to be reviewed should be the first step in the business process re-engineering project. In defining specific areas for improvement, the organization can ensure that the effort focuses on value and customer requirements.
As organizations work to drive time and cost out of business processes, they often turn to technology as a solution. The advent of new technologies such as the Internet has allowed organizations to rapidly bring new capabilities that dramatically improve business processes. The availability of new technologies and the drive for rapid implementation could put the organization at risk by driving key controls out of improved business processes and lacking key controls in new business processes. An IS auditor should always make sure that a re-engineered business process has not inadvertently removed key controls from the previous control environment.
The implementation of BPR affects the culture, structure, and direction of the organization. Generally, the largest impact of re-engineering is on the staff. The organization should have a change-management process and teams that can evaluate possible issues or problems that might arise and that can provide solutions. The change-management team should monitor the re-engineering process to ensure that it is meeting the strategic plan and goals of the organization. As the re-engineering is implemented, the organization should see improvements in products, services, and profitability. The proper implementation of technology should reduce manual intervention and controls, producing an accelerated production and delivery of products and services.
A couple emerging business and technology trends illustrate these improvements. The first is customer relationship management (CRM), which focuses on managing detailed customer information. This might include previous transactions and customer requirements, allowing organizations to match customer needs to products and services. A CRM system usually integrates a database, web technologies, telephony, accounting, and fulfillment systems. This integration enables organizations to capture transaction data, customer preferences, order status, and demographic information. This gives an organization a complete view of its customers across all business units and product lines, and enables it to proactively identify which products or services the customer might need.
The second, supply chain management (SCM), is the improvement of an organization’s product and service design, purchasing, invoicing, distribution, and customer service. The implementation of SCM involves streamlining the supply chain through the collaboration of entities in real time and the realization of just-in-time (JIT) delivery. JIT delivery reduces the overall cycle time associated with manufacture and inventory by creating products and services based on customer demand.
One of the technologies associated with SCM is the process of electronic funds transfer (EFT). EFT is an electronic payment process between buyers and sellers that is very efficient because it reduces paper transactions and manual intervention.
After an organization has developed a strategic plan and defined its goals, it must measure its progress toward these goals. Key performance indicators (KPI) are quantifiable measurements that are developed and accepted by senior management. Key performance indicators vary by organization but are created as long-term measurements of an organization’s operational activities against its goals. The organization uses quantifiable measurements that ensure the measurement of expected outcomes as opposed to activities. As an example of a goal, the IT organization would expect to deliver services in accordance with service-level agreements (SLA). The IT organization would measure actual service levels against the SLA, identify gaps, and define controls to proactively reduce the service-level failures to meet the SLA.
Some organizations tend to measure things that are easy to measure instead of those that are critical to the organization meeting its goals. These types of measurements might include the number of events but not the expected outcome from the events. To ensure that KPIs are understandable and do not detract from the organization’s mission, they should be kept to a minimum of three to five. The use of KPIs provides management with a compass that allows for course corrections in meeting organizational goals and a communication tool for the entire organization defining the importance of achieving these goals.
Another way to measure organizational performance is the balanced scorecard. The balanced scorecard is a management tool that clarifies an organization’s goals, and defines actions and the measurement of those actions to meet goals. The balanced scorecard differs from previous methodologies, in that it combines measurement of all business processes. This allows managers to see the organization from many different perspectives and identify areas for improvement. The balanced scorecard incorporates measurements of financial performance, customer satisfaction, business processes, and the capability to improve business processes. ISACA defines the application of the balanced scorecard to IT as a three-layered structure that addresses the four perspectives through the following.
Mission:
To be a preferred supplier of information systems
To deliver effective and efficient applications and services
To obtain reasonable business contribution of IT investments
To develop opportunities to answer future challenges
Strategies:
Use preferred suppliers of application and operations
Foster user partnerships and greater customer service
Pursue efficient and economical developments and operations
Control IT objectives
Provide business value to IT projects
Provide new business capabilities
Train and educate IT staff, and promote excellence
Provide support for research and development
Provide a balanced set of metrics to guide business-oriented IT decisions
Table 7.1 integrates the ISACA example and shows some possible measures associated with a balanced scorecard.
Table 7.1. Balanced Scorecard Perspectives, Objectives, and Measures
Chapter 6, “Business Application System Development, Acquisition, Implementation, and Maintenance,” discussed the review of general controls, which include the controls over project and change management, systems development, operations and maintenance, and network operations. This section discusses application controls, which relate directly to the functions (input, processing, and output) performed by applications. Application controls are used to ensure that only accurate, complete, and authorized data is entered into a system. These controls can be either manual or automated and ensure the following:
Valid, accurate, and complete data is entered into the system.
Processing of data is accurate and performs the function(s) it was created for.
The processing of data and results meet expectations.
The data is maintained to ensure validity, accuracy, and completeness.
Manual controls include checks performed by IT staff and IS auditors such as the review of error logs, reconciliations, and exception reports. Automated controls include programming logic, validation and edit checks, and programmed control functions.
ISACA recommends that the testing of automated controls include the use of manual procedures to ensure proper investigation of exceptions and that the IS auditor’s tasks include the following:
Identifying application components and transaction flow through those components, and gaining an understanding of the system by reviewing system documentation and performing interviews
Applying the appropriate audit procedures to test control strengths and weaknesses, and evaluate the impact of control weaknesses
Analyzing test results and audit evidence to determine whether the control objectives were achieved within the control environment
Ensuring the application’s operational effectiveness and efficiency by comparing the application with efficient programming standards, and comparing systems functionality to management objectives for the system
The IS auditor should use a combination of manual review (system documentation and logs), observations, integrated test facilities, and embedded audit modules. The IS auditor must review application controls, data integrity controls, and controls associated with business systems and components. These components might include electronic data interchange (EDI) and electronic funds transfers (EFT).
In reviewing application controls, the IS auditor should review the following areas:
Input/output controls
Input authorization
Batch controls
Processing control procedures
Processing
Validation
Editing
Output controls
Critical forms logging and security
Negotiable instruments logging and security (signatures)
Report distribution
Balancing and reconciliation
Output error handling
Output report retention
An IS auditor must first understand relative business processes before performing an application audit. This can be accomplished by reviewing the business plan, the IT strategic plan (long and short term), and organizational goals.
In auditing input and output controls, the auditor must ensure that all transactions have been received, processed, and recorded accurately, and that the transactions are valid and authorized. The auditor should review access controls and validation and edit checks. It is important to remember that in an integrated environment, the output of one system could be the input to another system. Input/output controls should be implemented for both the sending and receiving applications.
Input can be either automated or manual, and it ensures that only authorized transactions are entered into the system for processing. Manual controls can include reports generated by the system that list transactions requiring manual authorization or source documents containing signatures. Some systems employ an automated control to provide authorization for data exceptions. An example is a sales transaction in which the price of the product is being reduced. The salesperson might not be authorized to reduce the price, but an automated request could be sent to a supervisor. The supervisor would then log in with a second-level password to authorize the price change.
When using manual controls, the organization must ensure that all documents are controlled and that procedures exist to ensure that they have been accounted for. Automated access controls include the following:
A batch control transaction summarizes totals of transactions within a batch. This transaction can be based on monetary amount, total items, total documents, or hash totals. These totals can be compared to the source documents to ensure that all items have accurate input. In addition, control totals ensure that the data input is complete and should be implemented as early as data preparation to support data integrity. Hash totals are generated by selecting specific fields in a series of transactions or records. If a later summation does not produce the number, this indicates that records have been lost, entered or transmitted incorrectly, or duplicated.
Data validation is used to identify errors in data regarding completeness, inconsistencies, duplicates, and reasonableness. Edit controls perform the same function as data-validation controls but are generally used after data has been entered but before it is processed. Table 7.2, created by ISACA, describes edit checks.
Table 7.2. Data-Validation Edits and Controls
During the review of input processing, the IS auditor can compare the transaction journal to authorized source documents. The transaction journal records all transaction activity and provides the information necessary for detecting unauthorized input from a terminal and completeness of transactions.
Processing controls ensure that data is accurate and complete, and is processed only through authorized routines. The processing controls can be programmed controls that detect and initiate corrective action, or edit checks that ensure completeness, accuracy, and validity. Processing controls also include manual controls, such as these:
Manual recalculation—. Periodic sample transaction groups can be recalculated to ensure that processing is performing as expected.
Run-to-run totals—. These verify data values throughout the various stages of application processing. They are an effective control to detect accidental record deletion in transaction-based applications.
Output controls ensure that information resulting from processing will be delivered in a consistent and secure manner to authorized persons. Per ISACA, output controls include the following:
Logging and storage of negotiable, sensitive, and critical forms in a secure place. These types of forms should be properly logged and secured to protect against theft or damage.
Computer generation of negotiable instruments, forms, and signatures. This type of output should be properly controlled. The organization should enable logging to provide a detailed listing of generation, and it should be compared with the forms received.
Report distribution. Reports can be distributed manually or automatically, but they should follow authorized distribution procedures. All reports should be logged. When automatic reports are distributed either electronically or to print devices, access-control procedures should ensure that only authorized personnel have access to the reports.
Balancing and reconciling. All output from applications should be logged via transaction logs. The output should be routinely balanced to control totals.
Output error handling. Procedures should exist for the controlling, logging, and reporting of output errors. The output transaction originators should be notified of errors in a timely manner for review and error correction.
Output report retention. Policies and procedures regarding record retention should be adhered to. The retention policy should ensure compliance with any legal regulations.
Data is stored in the form of files and databases. Data integrity testing ensures the completeness, accuracy, consistency, and authorization of data. This differs from application testing because it tests data that is stored within a system after input and processing. The testing of stored data might uncover weaknesses in input/output or processing controls. Two types of tests are associated with data integrity:
Referential integrity tests—. Referential integrity works within a relational data model within a database and ensures that the relationships between two or more references are consistent. If the data in one reference is inserted, deleted, or updated, the integrity to the second reference is maintained through the use of primary and foreign keys.
Disabling referential integrity controls can result in invalid transactions, such as a payment to a vendor that is never recorded to the vendor payment database.
Relational integrity tests—. These tests ensure that validation (either application or database) routines check data before entry into the database.
The purpose of EDI is to promote a more efficient and effective data-exchange process by reducing paper, errors, and delays. In using EDI, organizations with dissimilar computer systems facilitate the exchange and transmittal of information such as product orders, invoices, and business documents. This is accomplished through standardizing the format of data and transmitting the data between systems. Organizations must ensure proper authentication techniques for sending and receiving data between EDI systems, to prevent unauthorized transactions. Transaction authorization is a primary security concern in EDI environments.
Traditionally, EDI systems contain the following components:
Communications handler—. A process for transmitting and receiving electronic documents between trading partners via dial-up lines, the Public Switched Telephone Network (PSTN), multiple dedicated lines, or value-added networks.
EDI interface—. This manipulates and routes data between the application system and the communications handler.
Applications interface—. This moves transactions to or from the application systems and performs data mapping. The EDI interface can ensure the validity of transactions and trading partners by checking information against a trading partner master file. After validation, the EDI interface generates and sends a functional acknowledgment. A functional acknowledgment is a message transmitted from the receiver of an electronic submission to the sender; it notifies the sender that the document was received/processed or was not processed. Functional acknowledgments provide an audit trail for EDI transactions.
When reviewing an EDI environment, it is important to remember that the EDI environment consists of software that transmits, translates, and stores transactions for processing. Network environments can often add to the complexity of program-to-program communication, making application systems implementation and maintenance more difficult.
Organizations that exchange data via EDI should have a trading partner agreement. The trading partner agreement defines the responsibilities of both organizations with regard to the handling and processing of transactions. The IS auditor should ensure that all transactions are accurately sent and received, translated, processed once, and accessed by authorized parties.
ISACA recommends the following tasks for the IS auditor in reviewing EDI controls:
Inbound EDI transactions that use public Internet infrastructures should utilize encryption to ensure confidentiality, authenticity, and integrity.
All inbound EDI transactions should be logged. Edit checks should be used to identify erroneous, unusual, or invalid transactions.
Inbound and outbound transaction message counts (sent/received) should be logged and periodically reconciled between trading partners.
Outbound EDI transactions should be compared to the trading partner master file before transmission, to ensure that transactions are being sent to authorized trading partners.
Authority to initiate, authorize, and transmit transactions should be properly segregated.
In evaluating business process change projects, the IS auditor should ensure that the change efforts meet the goals defined in the strategic plan, as well as the culture of the organization. The IS auditor should ensure that the organization has clearly defined the areas for review, and developed a project plan organization that will use proper project and change management processes.
The project plan should ensure that the goals of the business process re-engineering effort are met and fulfill organizational goals. The project should have a detailed plan with an assigned project manager who is experienced with business process re-engineering. A change-management team and plan should be established with mitigation plans for possible issues or problems during the re-engineering effort. The change-management team should assist the staff in transitioning to the re-engineered business process, as well as monitoring the project’s progress toward the re-engineering goals and strategic plan.
As stated in the section “IS Project-Management Strategies and Policies” in Chapter 2, “Management, Planning, and Organization of IS,” the IS auditor should look for the following risk indicators when auditing the business process re-engineering project:
BPR project leaders have insufficient domain expertise.
BPR project teams are unqualified to handle project size/complexity.
BPR project team members are dissatisfied.
The BPT project does not include input from all affected parties.
BPR project recipients are dissatisfied with project outcomes.
IT governance encompasses the information systems, strategy, and people. This control helps ensure that IT is aligned with the organization’s strategy and goals. The board of directors and executive officers are ultimately accountable for functionality, reliability, and security within IT governance.
Within the IT governance structure, there should be clearly defined roles and responsibilities. The IT department should implement best practices in its operational and development methodology and should have a structured approach to project and change management. Overall, the IT governance structure ensures the efficient and effective use of resources in the secure and reliable deployment and maintenance of information systems.
An important area of IT governance is risk management. Risk management is the process that enables IT managers to balance the operational and economic costs of protective measures, and achieve gains in mission objectives by protecting the IT systems and data that support business objectives. In the development of a risk-management plan, ISACA states that the organization must do the following:
Establish the purpose of the risk-management program. In establishing the purpose for the program, the organization will be better prepared to evaluate the results and determine its effectiveness.
Assign responsibility for the risk-management plan. To ensure the success of the risk-management plan, the organization should designate an individual or team responsible for developing and implementing the risk-management plan. The team should coordinate efforts across the organization in identifying risks and defining strategies to mitigate the risk.
As stated in Chapter 1, “The Information Systems (IS) Audit Process,” risk can be defined as the possibility of something adverse happening. Risk management is the process of assessing risk, taking steps to reduce risk to an acceptable level (mitigation), and maintaining that level of risk. In developing the risk-management plan, the organization should identify organizational assets as well as the threats and vulnerabilities associated with these assets. After identifying potential vulnerabilities, the IS auditor should perform a business impact analysis (BIA) of the threats that would exploit the vulnerabilities.
The IS auditor can use qualitative or quantitative analysis during the BIA to assess the potential impacts, or degree of loss, associated with the assets. Quantitative impacts are easily measured because they can result in a direct loss of money, opportunity, or disruption. Qualitative impacts are harder to measure because they result in losses associated with damage to reputation, endangerment of staff, or breach of confidence. In other words, a quantitative approach attempts to assign real numbers to the cost of threats and the amount of damage, whereas a qualitative approach uses a ranking method to analyze the seriousness of the threat against the sensitivity of the asset.
When the BIA is complete, the organization must determine whether the risk is acceptable. If not, the IS auditor can evaluate the existing controls or design new controls to reduce the vulnerabilities to an acceptable level of risk. The controls, called countermeasures, can be actions, devices, procedures, or techniques. After the organization has applied controls to the asset, the remaining risk is called residual risk. The organization’s management sets acceptable risk levels; if the residual risk falls below that level, further controls are not required. The IS auditor can evaluate this control to see whether an excessive level of control is being used. The removal of excessive controls can result in cost savings to the organization. The organization’s acceptance of residual risk takes into account the organizational policy, risk-management plan and measurement, and the cost-effectiveness of implementing controls.
The risk-management process provides management with an effective method of understanding risk and achieving a cost-effective balance when applying countermeasures. The risk-management program must be supported by senior management and must have a designated individual or team to be successful.
In most organizations, the executive director works with the board of directors to define the purpose for the risk-management program. In clearly defining the risk-management program goals, senior management can evaluate the results of risk management and determine its effectiveness. The risk-management team should be utilized at all levels within the organization and needs the help of the operations staff and board members to identify areas of risk and to develop suitable mitigation strategies.