Some managers seem preoccupied with making better use of current data—internal operations data, customer/client/patient data, supplier data, and market data to name a few data sources. Major data challenges are many including the increasing volumes of varied data, mixed data quality, data security, changing data regulations, generating insights from data, using analytics better, and identifying new opportunities to derive value from data. At the heart of implementing a successful digital transformation strategy is solving two key challenges: (1) managers must understand the value of current data and existing data sources, and (2) managers must have appropriate technology tools, skills, and techniques to support the digital vision. Successful transformation is about more than using current data.
Now let’s begin to tackle these challenges to effective data-based decision making and implementing a digital transformation strategy. The following five sections of this chapter discuss: (1) different approaches to managerial decision making including data-based, data-driven, and data-informed, (2) the need for computerized decision support, (3) key skills required for data-based decision making, (4) steps for making data-based decisions, and (5) the importance of data-based decision making for gaining an organizational competitive advantage. To overcome data challenges, one must determine how data can be used to help improve managerial and organizational decision making processes and hence business outcomes. Some people advocate for data-driven decision making, others for data-based or data-informed decision making. The next section explores these differing approaches to using data in decision making.
Data-Based, Data-Driven, and Data-Informed Decision Making
Decision makers are confronted with evolving and expanding data resources and there is a pressing need to ask better questions to help solve real business problems. One can read a variety of case studies about how using data can improve decisions in many domains including education, retail sales, health care, and financial services. Using more data and analytics is often identified as the key to success in these situations.
A number of phrases have been used by authors and consultants to describe the increasing use of data to improve decision making in organizations. The word data is modified as data-based, data-driven, or sometimes data-informed decision making. These phrases are often used interchangeably to refer to an improved organizational decision support capability. While the terms are related, there are important differences. After reviewing prior usage, we find it most useful to focus on data-based decision making. We consider each of these concepts as mutually exclusive, but in some situations complementary. For managers to meaningfully engage with data opportunities and challenges they need to understand how these decision making approaches can be formulated, managed and exploited as part of an organizational digital transformation strategy. Let’s examine these three approaches to using data in decision making.
Data-based decision making refers to an ongoing process of collecting and analyzing different types of data to aid in decision making (Power 2017). Decisions are based on data facts, values and vision, intuition, and ethical guidelines. Data-based decision making usually incorporates many diverse data types from a variety of sources including quantitative data balanced with “softer” data that is more descriptive in nature. Data-based decisions are primarily based on data, but analysis and judgment are also very important. Ethical decision making should be incorporated in data-based decision making. Decision makers should apply moral rules, codes, or principles to guide choices for right and truthful behavior.
Data-driven decision making or data-driven management is widely used in articles, consultant reports, white papers and more recently in academic research papers to characterize a particular type of decision making. Data-driven decision making refers to the collection and analysis of data to make decisions, but the data determines the action. Data “drive” the decision making and decisions are made using verifiable data. Some consider data-driven as synonymous with business intelligence, while other authors link the phrase to decision automation. Provost and Fawcett (2013) define data-driven decision making very broadly as “the practice of basing decisions on the analysis of data rather than purely on intuition.” According to a number of sources (McAfee and Brynjolfsson 2012; Frick 2014), organizations that use “data-driven decision making” are more productive and more profitable than their competitors. In these research studies respondents likely had various understanding of data-driven decision making.
Data-driven, data-informed or fact-based decision making means managers use and evaluate data to make decisions, but providing more data is not necessarily the way to improve decision making effectiveness. As a concept, data-driven decision making is often used in conjunction with big data and data analytics, particularly quantitative/statistical analytics.
Data-informed decision making is a term used when data and facts are an influential factor in decision making, but not the only factor. According to Maycotte (2015), decisions are complex phenomena that require significant human input in terms of experience and instinct. Maycotte believes, he believes that decisions should not be purely “driven” by data but data may be used to support experienced decision makers to be faster and more flexible in their decision making. He advocates that decision makers need to “strike the balance between expertise and understanding information.” The U.S. Department of Education prefers the terms data-based or data-informed decision making over data-driven decision making asserting that decisions should not be based solely on quantitative data.
Using data in decision making should be contingent on the decision situation. Figure 3.1 suggests a continuum of decision situations ranging from highly structured and routine to highly unstructured and nonroutine and recommends different decision making depending on the situation.
Figure 3.1 Using data in decision making
Data-driven decision making can be used effectively in highly structured situations when appropriate data and analytics are available. As a decision situation becomes more unstructured, the best one can do is data-based decision making because more qualitative content and subjective assessment is needed. It is appropriate and recommended to base a decision upon quantitative and qualitative data in these semistructured decision situations, but other factors like data quality, data relevance, and data timeliness must be resolved. Finally, in highly unstructured decision situations the best one can expect is data-informed decision making. In this process, one examines available data and tries to see how it informs ones understanding of a situation. In a highly unstructured decision situation, an effective decision maker needs both knowledge and facts. A subjective assessment and assumption analysis becomes especially important. Decisions related to implementing digital transformation involve semistructured and unstructured decision situations.
Managers need to develop and embed processes for collecting, storing, maintaining, and analyzing data that can help answer important, recurring decision questions. Creating and managing an approach to quantitative and qualitative data use requires a sophisticated information system. A mix of people skills, technologies, and managerial procedures are needed to create information and support its timely flow to decision makers for data-driven, data-based, and data-informed decision making. Using data appropriately in decision making is key to successful digital transformation.
Need for Computerized Decision Support
In this digital era, managers realize the growing need for computerized decision support. Today decision making is challenging and the challenge to make “good” decisions is increasing. The need to make faster decisions has also increased. Too much information is common in decision situations and much of that information is often only marginally relevant. Finally, there is often more distortion of information in society. For these reasons specifically, there is an emphasis on improving data-based decision making in organizations. These reasons also create a need for more analytics and decision support.
Overall, a complex decision-making environment creates a need for computerized decision support. Decision support remains a broad concept that prescribes using computerized systems and other analytical tools to assist individuals and groups in making decisions.
Research and case studies provide evidence that a well-designed and appropriate computerized DSS can encourage data-based decisions, improve decision quality, and improve the efficiency and effectiveness of decision processes.
Most managers want more analysis and they want specific decision relevant reports quickly. Certainly, managers have many and increasing information needs. Effective decision support provides managers more independence to retrieve and analyze data and documents to obtain facts and results when they need them.
From a different perspective, cognitive decision making biases exist and create a need for decision support. Information presentation and information availability influence decision makers both positively and negatively. Reducing bias has been a secondary motivation for providing analytics and decision support. Most managers accept that some people are biased decision makers, but likely wonder if proposed analytics and decision support would reduce bias. Decision makers do “anchor” on the initial information they receive and that behavior influences how they interpret subsequent information. In addition, decision makers tend to place the greatest attention on more recent information and either ignore or forget historical information. The evidence is convincing that these and other biases can alter decisions.
Changing decision making environments, managerial requests, and decision maker limitations create a need for more and better decision support. We should consider building a computerized decision support when two conditions exist: (1) Good data/information is likely to improve the quality of decisions, (2) potential decision support users recognize a need for and want to use technology to support their data-based decision making needs.
Key Skills of Data-Based Decision Makers
Decision making habits are often learned by trial and error. Decision making skills should be learned through more deliberate, systematic effort. Culture can promote the use of data to make decisions or be neutral on this topic. More data and easier to use analytical tools provide an opportunity for improving operational decision making, but many managers must learn new behaviors and skills to actually use data and analyses effectively. Generally, managers must expand their skill sets to use data and analysis effectively. Data-based decision making requires a specialized skill set in addition to other decision making skills.
Organizations that embrace measurement have a data-centric culture. This encourages and rewards managers for making decisions based on meaningful data, rather than solely based on intuition, cf., Kanter (2013). Managers must enhance and refine their understanding of the possibilities of data analysis. Managers must strive to understand the meaning of frequently used analyses. Also, managers must be rewarded for incorporating results of data and analysis into their thinking about a situation.
Shea, Santos, and Byrnes (2012) differentiate between data-driven and data supported decisions. They note both processes use quantitative and qualitative data to inform and make decisions. Supposedly data-supported decisions “use the same data but they also take into account people, issues, ethics, and broader system effects.” They caution that an excessive “data driven” emphasis can contribute to ethical blind spots poor decisions. Data-based decision making can and should incorporate ethics and ethical decision making.
Using data and analyses is sometimes challenging. Rob Enderle (2013; 2014), a technology analyst, provides examples of what he considered poor use of data and analyses at IBM, Microsoft, and Siemens. For example, he reports Microsoft’s internal market research organization was providing executives with “results that made decisions they had already made look smarter.” Hindsight can suggest data distortion and misuse, but based on his personal experience he observed “a surprisingly small number of the companies that sell analytics tools actually rely on those tools for major decisions.”
Blogger Kalie Moore (2014) at Business Intelligence software vendor datapine.com raises a similar issue. She writes “insights we provide are completely useless if, at the end of the day, these reports are ignored by the actual decision makers.” Moore felt business leaders were not using data in decision making for three reasons: (1) overreliance on past experience, (2) going with their gut and cooking the data, and (3) cognitive biases. These are serious concerns. There are ways to overcome biased behavior, but managers must become aware of their own biases and the problem resulting from specific biases. Managers must develop reflective skills, especially regarding biases in data use, to become effective data-based decision makers. Reflective skills means thinking about or reflecting on what you do.
Data analyses can be used to bolster and provide biased confirmation of previously made decisions. Also, analyses can be requested that support biased rationalizing of decisions. Skilled data-based decision makers must learn to reserve judgment and postpone a final choice until the available facts are presented and evaluated. A decision should then be made that incorporates and reconciles the facts.
So what are the specialized data analytics decision skills managers and decision makers need? The primary skills seem to broadly encompass: (1) collecting and identifying relevant data, (2) using software to perform statistical analysis including charting of data, (3) interpreting data and analyses in the context of an actual decision situation, and (4) using analyses of data, including sensitivity analyses, to inform decisions. Let’s review these skills briefly.
- Collecting and identifying relevant data. Organizations collect large amounts of data and external data can also be purchased. Often new data can also be captured. Managers need to understand data resources and data capture and how to work with stored data, to use metadata, to identify what data is available and what new data should be captured.
- Using software to perform statistical analysis including charting of data. Often desktop tools like Excel and Tableau are adequate. Learning a statistical analysis package helps decision makers interpret analytical results and understand limitations of statistical analysis.
- Interpreting data and analyses in the context of an actual decision situation. Decision makers need to match data and analyses to questions of interest. Many decisions can be framed either in terms of gains or losses. How a decision is framed can also impact choices. Decision maker ask: What do I need to know about this situation? Is there data that will help me understand my choices in the situation? What does the data mean? Do I have a pre-conceived solution or biases?
- Using analyses of data, including sensitivity analyses, to inform decisions. Data can inform decisions, but data does not always provide conclusive evidence. In some situations data analysis shows a strong correlation, but the causal evidence is much more circumstantial. Rather than ignoring data, managers should show caution when they use available data. Correlation is not causation, but in many cases correlation is the strongest conclusion about a relationship.
Many observers agree quantitative skills are important to data-based decision making.
Until recently, data analysis skills were primarily taught to statisticians, market researchers, actuaries and other specialists more than to people planning careers as managers. Times have changed and teaching applied data analysis skills is increasingly popular. Top International Business Schools are addressing this managerial skills gap. There is broad recognition that managers and decision makers need to be skilled data users and data interpreters. Managers need to be skilled at data-based decision making.
Using data and analyzing data is every manager’s job. If that goal is to be realized, then current and future decision makers must develop and enhance skills needed to use data effectively. Rationalizing before a decision is made or afterward is equally inappropriate. A skilled, data-based decision maker follows a process that begins with asking the right questions, and then answering the question using facts, relevant data and analyses prior to making a decision.
Steps to Develop Data-Based Decision Support
Encouraging and developing data-based decision support is an organizationwide effort and requires many resources, including people, money, and technologies. Building an effective enterprisewide decision support capability can help improve decision making, but meeting that goal is a challenging task. Providing companywide decision support requires creating a sophisticated information technology architecture of computing assets. That architecture provides the foundation for data-based decision making and digital transformation. Data-based decision making benefits from computer-based support for collecting, analyzing, and sharing different types of data. Often relevant decision support information is derived from real-time and historical quantitative and qualitative data.
Creating and managing a modern computing architecture requires a mix of people skills, technologies, and managerial procedures that are often difficult to assemble and implement. For example, storing a large quantity of decision support data is likely to require purchasing the latest hardware and software. Most companies need to purchase high-end servers with multiple processors and advanced database systems translytical databases NoSQL databases such as MongoDB, Cassandra, and CouchDB, and translytical databases to support real-time transaction processing and data analytics. To implement a decision support architecture an organization also needs people with advanced database design and data management skills.
How can managers increase the chances of a successful data-based decision support implementation? After evaluating some alternate suggestions, we have concluded that once an appropriate need has been identified, then the following steps can help create, implement, and promote use of data-based decision support in decision-making processes, see Figure 3.2.
Figure 3.2 Steps to create a data-based decision support capability
In Figure 3.2, the first step is to identify an influential project champion with a decision support need. The project champion must be a respected, senior manager. A project champion can deal with political issues and help insure that everyone realizes they are part of an analytics and decision support team. Managers need to stay focused on a company’s decision support development goals.
Second, managers should be prepared for technology shortfalls. Technology problems are inevitable with data-oriented decision support projects. Often the technology to accomplish a desired decision support task is not currently available or is not easily implemented. Unforeseen problems and frustrations will occur.
The third step is to tell everyone as much as you can about the costs of creating and using the proposed decision support capability. Managers need to know how much it costs to develop, access, and analyze decision support data.
Fourth, be sure to invest in training. Set aside adequate resources, both time and money, so users can learn to access and leverage the new decision support capability.
Finally, market and promote the new capability to the managers you want to use the system. Provide incentive and motivation for appropriate use of the system. Provide incentive and motivation for appropriate use of the system.
Effective decision support requires ongoing innovation and refinement. As decisions become more complex and as data increases in quantity and variety, systems must be refined and enhanced. Decision support requires an iterative development process, executing the steps in Figure 3.2 repetitively.
Find an Opportunity to Create
Competitive Advantage
Managers want to create capabilities that provide a competitive advantage. Decision support, analytics, and business intelligence can provide advantage, but the mere existence of a capability does not create sustainable advantage. According to Setia et al. (2013, p. 583) “the relative ubiquity of digital technologies implies that merely investing in digital technologies or enhancing their usage may not be sufficient for a firm to gain competitive advantage.” The real value comes from understanding how a proposed capability may provide advantage. This knowledge is crucial for evaluating opportunities. Also, it is important to assess how likely it is that a successful implementation will provide advantage. At an abstract level based on Barney (1991), technologists need to ask if the novel capability is valuable, rare, inimitable, and nonsubstitutable. Then establish whether the organization is ready to implement the capability.
A competitive advantage is a resource, capability, skill, or characteristic of an organization that significantly enhances success in a market, rivalry situation, or competitive encounter. Competitive advantage results from doing something better than competitors that create value and superior performance in a timely manner.
A proposed capability must have the potential to create real value in a realistic time frame. For example, a novel decision support capability must demonstrably improve decision-making efficiency or effectiveness or both. In general, a novel decision support capability should be hidden from competitors and it should be difficult for competitors to identify and emulate. An internal facing system used by managers is more likely to remain rare and unknown to competitors than an externally facing capability used by customers or other external stakeholders. A proposed capability must also be difficult to duplicate or imitate. Systems developed by company information technology staff are more likely to create sustainable advantage than off-the-shelf solutions purchased from a vendor. Finally, the functionality of any new capability must be difficult to duplicate if it becomes known. One hopes any substitutes for the capability have serious limitations.
Decision support and analytics may and can create competitive advantage. A decision support capability is a competitive advantage when it is (1) a major strength, (2) unique and proprietary, and (3) sustainable long enough to realize a payback. Digital transformation and improved decision support may not create a competitive advantage—it depends on the vision, competitor actions, timing, and implementation.
Barney’s resource-based view of a firm suggests questions that should be asked when vendors or internal advocates are promoting any capability as a potential competitive advantage.
- Will the proposed capability create significant value?
- Is the proposed capability novel and rare?
- Is the proposed capability difficult to copy, duplicate, or imitate?
- Do substitutes for the capability have limitations that will discourage using them?
- Is the organization ready to implement and exploit the proposed capability?
Decision support can create competitive advantage by significantly improving data-based decision making efficiency and effectiveness, by supporting cost and/or differentiation strategies, and by increasing organizational control, innovation, or adaptability. High risk decision support projects such as key digital transformation decision-related projects are the most likely to result in a competitive advantage or fail spectacularly. Gaining any advantage may require large financial investments and be temporary. Some decision support development opportunities are better than others. Many very useful decision support capabilities will not provide a significant competitive advantage, but the capabilities are needed to remain competitive in the industry. Evidence about the need and potential value of a proposed capability can help choose among opportunities.
Some software vendors claim a specific “new” application will provide a competitive advantage. Supposedly organizations that implement a vendor’s solution will gain a competitive advantage. This broad promise sounds too good to be true, so don’t believe the promise. Ask questions, get answers and facts, tailor and customize off-the-shelf applications.
Summary
More efficient and effective managerial decision making is difficult to achieve in an increasingly complex, data intensive, digital business ecosystem. Stakeholders, including shareholders, senior management, customers, and partners, have high expectations for better results and continue to want more value.
Increasing and improving data-based decision making is key to successfully implementing a digital transformation vision. Some people advocate for data-driven decision making, others for data-based or data-informed decision making. These approaches differ and each can provide value in the right situation. Data are important and decision making should incorporate facts, but often assumptions and opinions should also influence choices. Managers need to learn new analytical skills and an organization’s culture should reward data-based decision making. Overall, complex decision-making environments create a need for computerized decision support and for more sophisticated decision making.
Using data-based decision making provides a generalized opportunity to create an advantage as part of a digital transformation strategy. In general, managers should use data-based decision making for semistructured decision situations. Managers should use all relevant data in semistructured situations, and should consider ethical issues, values, situational factors, assumptions, and other less tangible factors. This factual, data-based approach is especially important in assessing digital transformation opportunities.