Chapter 12 Metrics-Based Decision Support Systems
Two Systems
Business information systems and metrics systems are interrelated but manifest a testy relationship. The metrics system attempts to address all the processes, while the archetypal information system focuses on short-term business results. The former provides a fertile ground for process research; the latter is anchored to issues related to delivery.
The two types of systems evolve with time and cut parallel inroads in the organization. It takes several cycles of experience and realization to bring them to unison. Until such time, the project information system is in better circulation. The fact that metrics has the potential of being a primary source of information is often missed.
As organizations mature and achieve integration, automated project dashboards are set up. These tools issue forth stereotyped reports, month after month, beating a track for metrics application, which will stifle other avenues.
The most natural environment for metrics is one of decision making. This involves creating alternative scenarios, doing what-if analysis, and generating models for simulation. Such features are beyond the scope of conventional project information systems. We need, therefore, to develop a metrics-based decision support system.
The Humble Beginning
In the beginning of the evolutionary history of metrics, the metrics system was regarded with skepticism and caution, and was considered an untrustworthy supplier of information. In those early times, the organization could not take a risk by depending on information from metrics systems that suffered from delinquencies. The business information system, which we denote by MIS, existed in any of its many forms as a strong protagonist. The business tolerated inadequacies in MIS, the familiar buddy but
225
Exhibit 1. A simple information environment.
quenched metrics system, the newcomer. The information scenario (Exhibit 1) was simple, with just these two elements, one well imbibed and the other awaiting admission.
In those beginning days, strangely, there was no next to automation in MIS. The project manager personally created MIS reports, collecting information from memory and through various sources.
Metrics systems, quite understandably, were handled by Quality Assurance (QA) or SEPG who furnished the periodical product health reports and, in tune with CMM, published process capability baseline reports.
MIS was instantly associated with the direct execution of business, and commanded utmost respect and attention. The metrics reports were put in circulation and were studied with detached interest but failed to excite energetic managerial action.
Advent of Software Management Tools
The information scene changed continually as more tools were introduced in the organization.
Defect tracking tools have total control of defects data and produce standard reports. Some defect tracking tools focus on software testing and maintain complete history since testing, while some start from the first review. The defect tracking tools also support certain defect metrics; the list could vary from just a few numbers to several scores. Defect data analysis likewise could vary from elementary control charts to sophisticated pattern recognition.
Project management tools help in planning and tracking the project using core metrics such as cost and schedule. They also contain vital data related
Exhibit 2. Heterogeneous information environments.
to work breakdown structure, task dependencies, responsibility allocation, critical tasks, completed tasks, earned value, and resource balancing.
Each tool operates from its own design philosophy, collects its own data, and generates very specific metrics. The organizational metrics database now has to gather data from a heterogeneous set of tools and their captive databases. Exhibit 2 illustrates the scenario.
Software Management Tools that Focus on Engineering
Some software management tools place emphasis on software engineering, support, and control, and cover the life cycle processes from requirements management to testing. These tools come as a suite of products or modules, each devoted to a life cycle phase and generate metrics for control and management.
The requirements management tools use case development, business modeling, and data modeling. They maintain a requirements repository and address the problem of communicating customer requirements to all stakeholders in the enterprise. They help in analyzing and tracing requirements.
Development toolkits include visual modeling, design, and runtime analysis. These tools provide support in automatic code generation and simulation. They provide the environment for development.
Software configuration management tools help in managing software changes through comprehensive version control and defect and change tracking.
Software Management Tools that Focus on Estimation
Another class of software management tools focus on estimating, tracking, benchmarking, and metrics analysis. These tools support quantitative approaches in software management and some of them enable statistical process control and forecasting. A more visible metrics plan is used by such tools. Metrics analysis, therefore, becomes a management tool for tracking the critical aspects of the project.
Software Management Tools that Focus on Testing
The third type of software management tools centers on testing. These tools help to measure product quality and manage defects. They quite naturally track defects in all life cycle phases and exercise a firm control on defect fixing and change management.
Dashboard
The three categories of tools support a management dashboard. Although the structure and the contents of the dashboard vary, this attempt to connect to management is an attractive and desirable feature of these products.
Software Management Tool Vendors
Following is a representative list of vendors who provide software management tools:
Birth of Process Databases
Measurement is an inseparable part of process; hence, there is a logical need for maintaining process metrics data in cognitive, semantic, and quantitative forms. The process owners — individuals or teams — are rich storehouses of metrics data. Certainly they commit the data to memory. Data entrenched in human memory needs to be gathered and archived before it evaporates. Discipline in personal and team processes, as envisioned by
Exhibit 3. Metrics in distributed information environment.
Humphrey, makes this happen. People record their experiences as numbers or notes, thus creating a process database.
The newfound metrics culture is responsible for the creation of a multitude of process databases across the organization. These metrics databases are highly localized, each denoting an individual style. To understand and later integrate them, we can group these diverse databases under some broader titles according to organization structure. To manage diversity in location, we can think of a distributed database, as indicated in Exhibit 3. The organizational metrics database is not an isolated information system but, by being seamlessly connected to the several process databases, is a network.
With optimum choice of the following elements, it is possible to overcome distribution difficulties and ensure a metrics database that is safe and consistent:
- Data structure
- Partitions
- Replication
- Redundancy
- Access rights
Proper distribution solves technical problems in sustaining process databases in different locations, making it possible to reap some inherent benefits of having local databases. As authors of data as well as the database, process owners tend to become metric owners. In them, metrics data analysis is an inspired activity.
Enterprise Integration
Intelligence Integration
A remarkable benefit of metrics systems is that they achieve integration of intelligence by creating a network of the decision centers. In the name of metrics data collection, analysis, and research, we really look into process behaviors from a single window. Study of interrelationships between metrics is indeed a study of possibilities of process integration. Publishing capability baselines all together in one edition brings to a common platform several business issues, from goals to performances. Enterprisewide intelligence integration through metrics is an easy win, a most natural, often unintended result replete with benefits.
Enterprise Resource Planning (ERP) and Metrics
Integrating the enterprise at the operational and tactical levels of management is achieved by enterprise resource planning (ERP) solutions. ERP enables the integration of data and business processes throughout an organization. Modern ERP solutions are supported by new technologies in information processing and networking. Internet technology has brought in further capabilities to the ERP and has made decision making possible at lower levels of management.
With enterprisewide IT solutions in place, metrics must be seen as an information product, one of the several beneficial results of IT implementation. The job of metrics data collection is totally replaced by metrics extraction from the operational databases. This scenario provides an ideal opportunity for automatic metrics creation and application.
When ERP and metrics system coexist, special attention must be paid to data collection. The data gathering processes may compete or duplicate, with unpleasant consequences. People may find they feed the same data to both the ERP software and the metrics system in two different points of time; or, essentially the same data may be collected in two different formats by the two systems. Sometimes, without being aware that the ERP modules generate some metrics, people work separately and generate metrics, perhaps because those ERP-generated metrics are hidden within the ERP system.
In classical software development projects, where software products are designed from basic concepts, the challenge of enterprise integration has been overwhelmed by problems in software engineering. Software development tools dominated the scene, and ERP implementation began tentatively in the Finance Department and expanded to human resource management (HRM). In this scenario, software development tools bred automated software metrics.
In the new generation of high-volume software projects focused on build, enhancement, and maintenance, the challenge has shifted from engineering to service, quality, and speed. The need has arisen for defining new business models and extending ERP to service management, supply chain management, project management, performance management, and quality management. When a complete suite of ERP modules is implemented, almost all metrics including the core ones can be extracted from the ERP databases.
By integrating business functions, ERP solution gives several benefits to the organization, including the few mentioned here:
- A single system to support rather than several small and different systems
- A single applications architecture with limited interfaces
- Access to management information unavailable across a mix of applications
- Access to best practice systems and procedures
- More integration hence lower costs
- More automation of tasks
- Increased flexibility
- Reduction of lead time
- Better customer satisfaction
- Improved performance
- Improved resource utility
On the other hand, when managers see the organization through ERP and its well-structured reports, they may soon recognize a problem: the system lacks an organic feel, tends to make things look more routine, and reduces sensitivity. Because of the richness of functionality, the “toy box effect” can take over.
A well-designed metrics system is an elegant and fitting complement to the ERP solution. The metrics system can make use of the ERP data and provide higher-level decision analysis, and function as an add-on decision module. The inherent nimbleness of such decision modules alleviates the stiffness of ERP. Metrics systems can be used to consolidate many benefits of ERP, and more importantly, avert some of the undesirable traits of ERP.
Enterprise Metrics
The goal of bringing a metrics culture in all business functions is strengthened by seeing metrics with ERP, where possible. When the metrics system involves a progression of values adding transformations of data into wisdom, ERP provides the raw material — data — which can be accessed using a single application from all kinds of business functions including the following:
- Finance
- Treasury
- HRM
- Enterprise controlling
- Investment management
- Production planning
- Manufacturing and production planning
- Sales and distribution
- Plant maintenance
- Quality management
- Materials management
- Project management
- Supply chain management
- Front office
- Performance measurement
- Service management
- Procurement management
- Payroll
- Utility
In all these functions, metrics and models can be created, establishing quantitative process management. Applying metrics in an ERP environment benefits from the groundwork already done by the organization in implementing ERP. Implementing IT in all these business functions lays the foundation for implementing metrics in these functions.
ERP Vendors
There are ERP solutions available that offer a wide range of capabilities to address a variety of business segments. There are also a host of ERP consultants who offer support in implementing ERP. Following is a partial list of ERP vendors:
- SAP AG
- Oracle
- JD Edwards
- Peoplesoft
- Baan
- SSA
- JBA
- Marcam
- Intentia
- QAD
- Ramco
Exhibit 4. Metrics and the information environment.
Process Intelligence
A new dimension emerges in the use of metrics databases when it is well fed by a rich information environment, as shown in Exhibit 4. The metrics database now becomes a large storehouse of assorted data pouring in from the various sources. The value of this data goes beyond providing information. The data contains intelligence that needs to be extracted for the ben-efit of decision makers. Extraction of this intelligence is a very delicate process and usually involves pattern recognition techniques and other sophisticated statistical methods. All these approaches are bundled into a methodology known as data mining.
Metrics Warehouse
The metrics database in its original form cannot be easily used for data mining. We have to create a data warehouse where the raw data is cleaned and structured to facilitate advanced treatment. Setting up a data warehouse is in fact preparing for the knowledge discovery process. It involves selection of data from various databases, cleaning the data, and removing erroneous and false data. Inconsistency and duplication of data will also be similarly removed. As a data warehouse is designed for decision-support queries, data that is needed for decision support is extracted from the operational data and stored in the warehouse.
The metrics warehouse structure should be time dependent, nonvolatile, subject oriented and integrated. The data will also be regularly enriched; new information will be continually added to the old, regardless of the sources. Thus the data warehouse can handle heterogeneous data inflow.
It may be noted that we are not recommending a data warehouse because of the volume of data involved. We need to use the data warehouse structure for its well-known benefits. Many SEPG members who have analyzed metrics data for publishing process capability baselines recall that they have manually cleaned, validated, and enriched data, the same way as
the data warehouse does. It makes practical sense to adopt data warehousing methodologies.
Metrics Data Mining
In data mining, we use computers to look at data and analyze it as the human brain does. Data mining is one of the forms of artificial intelligence that uses perception models, analytical models, and several algorithms to simulate the methods of the human brain. This would suggest that data mining helps machines to take human decisions and make human choices. The user of the data mining tools will have to teach the machine rules, preferences, and even experiences in order to get decision support.
With a metrics warehouse in place, we can install data mining algorithms, such as:
- Query tools
- Statistical techniques
- Visualization
- Online analytical processing
- k-Nearest neighbor
- Decision trees
- Association rules
- Neural networks
- Genetic algorithms
However, it is almost impossible to design universally applicable data mining algorithms. But we select from a large number of commercially available data mining tools to suit the purpose at hand.
Applying Business Intelligence Tools to Metrics
These tools (such as from www.spss.com) analyze data and create business intelligence using data mining techniques. These tools offer an impressive array of features, including:
- Event detection
- Scorecard creation
- Ad hoc queries
- Creation of data marts
- Model building
- Model exploration
- Experiments with models
- Iterative learning
When applied to metrics data, these tools create process intelligence.
Metrics applications now become automated and fast.
Enterprise Intelligence Systems and Metrics
The new generation of ERP solutions provide interfaces to related technologies such as the following:
- Business process reengineering (BPR)
- Decision support systems (DSS)
- Executive information systems (EIS)
- Data warehousing
- Data mining
- Analytic intelligence
- Online analytical processing (OLAP)
- Supply chain management
- Customer relationship management
- Balanced scorecard
- Value chain
- Activity-based management
- Human capital management
- Operations management
- Knowledge management
- Risk management
These applications may be from different vendors and may not mesh. Integrating all these applications may require some effort and the use of enterprise application integration (EAI) techniques.
Some vendors offer product suites (such as www.sas.com) with many of the applications we have listed, from which we can easily build integrated intelligence systems. Equipped with such power, metrics data analysis and generating process intelligence become very simple tasks.
A Symbiotic Dependence
With the usage of software management tools, process databases, ERP solutions, and enterprise intelligence systems, the organization generates large volumes of data. The metrics system faces a new problem of having to cope with a complex information environment, comprising assorted data generators and their prolific creations. With time, the captive databases attached to various tools grow in size. This explosive growth of enterprise data might not proportionately enrich the metrics database of the organization.
Until the enterprise data reaches a common metrics database, the usage of data happens to be isolated attempts in narrow areas. But the organization fails to gain global perspectives or benefit from internal benchmarking. Integrated project management using quantitative methods remains an even more difficult proposition.
Exhibit 5. Metrics and automated DSS.
The relationship between the metrics system and its complex information environment, indicated in Exhibit 5, is symbiotic with mutual advantages. The metrics system should be made more organic and flexible, and should be in a position to adapt itself to fit into the complex environment.
Being organically coupled to a complex information environment offers new and changing roles to the metrics system. As information provider, the metrics system furnishes data; as a “rider,” it extracts data, although partially, from the several databases. A metrics system operates, in this context, as an information exchange bureau, providing opportunities for metrics data conduits across the organization.
The structure of a metrics system must closely follow the evolution of the IT environment in an organization. The interaction between metrics and information systems is so complex that IT strategies will shape metrics strategies.
An Economic Alternative: Metrics-Based Decision Support Systems (DSS)
Metrics-Based DSS
The IT approach to DSS involves complex tools and sophisticated statistical techniques. A simpler approach is to build DSS from metrics. The architecture of a metrics system is conceived around decision centers, and directly addresses the question of quality of decision making.
Constructing a metrics system, well balanced in its metrics choice and equipped with analytical capabilities, amounts to construction of a metrics- based DSS (MBDSS).
Human-Centric Approach
A metrics system provides a low-cost DSS by sustaining process models, permitting analysis of models, and forecasting. By extending metrics to support processes, the scope of model-based analysis is also extended to other than the core processes.
To create process intelligence from metrics is a “human” activity. From available metrics, by human inquiry and creativity, we can generate meaningful models and carry out decision analysis. Human analysis allows higher-level decision analysis in “unstructured” modes, whereas analysis by common tools can solve only structured problems. Human analysis possesses strategic decision-making capability.
Manual Analysis
MBDSS provides decision support using simple and manual methods. The first is a process capability baseline, extracted from a metrics database. Other simple tools, which can work from either a metrics database or more efficiently from a metrics warehouse, are:
- Data visualization techniques
- Frequency domain analysis
- Time domain analysis
- Relationship domain analysis
- Process synthesis
These data analysis methods are described in Chapter 4 through Chapter 8. Using these, we can build a library of process models, each supporting the decision-making process.
Simple Tools
MBDSS requires some minimum IT support. The least is an RDBMS. Then we need some spreadsheet tools for statistical analysis.
MBDSS can be improved by adding a metrics warehouse and a statistical analysis package. There are a large number of data analysis tools that provide basic statistical analysis. We have listed in Exhibit 6 a few tools that support process control. Typical features of these tools are:
- Process capability analysis
- SPC
- Design of experiments
- Business intelligence options
- Real-time monitoring
- Report generation
Exhibit 6. Data analysis tools.
| S.No. | Tool | Vendor | Address |
| 1 2 3 4 5 6 7 8 |
Analytica XLReporter origin, origin pro DADISP/2000 QI Analyst4.2 DT Analyst2.0 SPSS Exact Tests Minitab |
Lumina SyTech OriginLab DSP wonderware wonderware SPSS Minitab |
www.lumina.com www.sytech.com www.originlab.com www.dadisp.com www.wonderware.com www.wonderware.com www.spss.com www.minitab.com |
Web Enabling
MBDSS requires an intranet for effective communication. The analysis tools can be made available in the network and accessed by the user for a “do-it-yourself” kind of analysis.
Knowledge Management
Creating process intelligence goes hand in hand with knowledge management initiatives in the organization. Experimenting with process models is a knowledge-generating game. Knowledge is in fact a byproduct of the metrics system in its natural course. In this context, a metrics system is a virtual process learning center (VPLC).
Human Inquiry
At the center of MBDSS is human inquiry, as shown in Exhibit 7. The process of human inquiry can have “boundary-less” access to all tools, databases, and knowledge management systems for decision making. If automated decision-making tools are installed, human inquiry would consider the decision prompts from such tools as inputs.
The final decision is human.
Metrics Dashboard
We can construct a simple metrics dashboard that presents pictorially the details about a chosen project or product. The dashboard can be designed using the drill-down approach and can have links to relevant applications.
Exhibit 7. Proposed metrics-based DSS solution.
MBDSS: Information, Intelligence, and Strategy
MBDSS contains all the necessary elements, connectivity, and human insight to help organizations develop strategies from data. It also gives practical alternative, less-costly approaches for intelligence creation. Above all, it makes the best use of metrics.