Business problems it solves

As knowledge workers, business users and analysts are tasked with discovering insights in the massive amounts of data that businesses collect. These insights can include identifying customer problems such as the following:

• Unexpected customer churn

• Customer relationship and management problems

• Subtle product issues such as returns and failures

• Price leakages due to excessive discounting

• Promotional failures

• Lost market share due to competitive actions such as aggressive pricing or a new product

Undetected and unaddressed, these problems can seriously undermine any business. Hence the task, the urgency, to find something—anything—in the data and take action.

Why you need it: expected business outcomes

There are two scenarios for the knowledge workers’ tasks:

Undirected exploration

Analysts are given vague or no guidance: no business problem or set of business questions. They are left on their own to use their own devices, imagination, and experience to “just discover something.” Consequently, they randomly explore, looking for anything, from potential product delays to spikes in returned orders to manufacturing slowdowns. They are free to formulate a conjecture given the lack of upper-level management guidance. Their data exploration is still based on an understanding and knowledge of the business, its problems, strategies, and market climate, all of which act as guides and impose biases and limitations on what is done.

Directed exploration

In this scenario, analysts have a clearly defined task given to them, such as to confirm or disprove an idea or address a specific problem. For example, the task might be to predict the effect on sales of a new promotional campaign or a competitive price action.

In either case, business users and analysts begin by becoming “data detectives” who survey existing complex, diverse, and sometimes legacy data systems (which can be siloed at different locations in the enterprise). They wrangle the data into a useful form and dig deep to locate potential trends and patterns for insight into problems, whether clearly defined or not. With the flood of data produced by modern businesses, they must do all of this in real time to avoid their data becoming stale and outdated soon after it is collected. Time pressures are high, and processes are complex.

Visual Data Discovery (VDD) plays a vital role in enabling knowledge workers, regardless of background and skill level, to be not just data detectives but knowledge seekers. They seek insight from data by visually and interactively drilling down in real time using interactive charts and images (see Figure 1-2) to examine data for insights into conjectures and problems, regardless of whether they are ill defined or well-articulated. VDD gives analysts the tools that enable them to blend datasets for the seamless and effortless exploration of data.

VDD offers knowledge workers the power of self-service discovery, finding patterns and relationships using their own ingenuity, with a deep set of diagnostic analytic capabilities freeing them to investigate potential relationships across a variety of structured and unstructured data sources. Coupled with the rise and advancement of self-service data preparation technology and technology-assisted analysis and pattern-detection platforms, VDD is now more relevant than ever for dealing with complex and disparate real-time data sources.

Figure 1-2. With Visual Data Discovery, business users are free to explore data for trends and insights to drive better decisions

The result of VDD is an agile, flexible, and scalable analytics deployment across the enterprise allowing for autonomous analytics for all knowledge workers. VDD tools are agile because of the high interactivity they provide and the resulting speed for identifying and addressing problems. Using VDD, knowledge workers can dynamically interact with visual images of their data, exploring different views for greater insight with little effort and, often, minimal technical training. Knowledge of the business and its driving problems surpass technical knowledge for analyzing data because technology-assisted analysis and pattern detection platforms will guide them to what is important.

The ability to visually interact with data in a self-service environment is fast becoming a vital element for business success. VDD is no longer an analytical convenience; it is a strategic necessity. It enables a flexible analytics deployment across the enterprise because it can handle any type of data from data warehouses, data stores, departmental data marts, and external data sources, as well as structured and unstructured data. The source and type are immaterial. The visualization tools are powerful enough to handle small as well as large quantities of data so that they can easily scale up to big data and cover the entire enterprise.

Key considerations, requirements, and capabilities

Self-service discovery must take into account the time these knowledge workers allocate to these tasks; that is, time that cannot be used on other tasks. There are, after all, only 24 hours in a day. This means that a VDD system must be very efficient and intuitive to use so that analysts can quickly, seamlessly, and effortlessly discover information and insight.

Which knowledge workers would use VDD tools? Business analysts, marketing directors, customer service managers, sales leaders and so forth with a good understanding of their domain, including business models, challenges, opportunities, risks, typical data, and processes. They are responsible for exploring data and generating insights.

All knowledge workers must be multifaceted and have an understanding of the business domain including its problems, strategies, structure, and market position. They must also understand the data collected including their source, structure, and issues.

Organizations have a greater likelihood of realizing the full benefits of VDD if they already exhibit a data-driven culture and follow best practices for technology that support the following:

• Data analysis workflow

  • Data access, data management, data preparation, ease of use for dashboard creation, interactive visual exploration

• Advanced analytics capabilities

  • Geoanalytics, predictive analytics

  • Scalability for complex analysis

• Publishing, sharing, governance

Example: An Airline¹

For its customers for prediction and proactive customer support, a well-known US-based airline needed a 360-degree view; a single end-to-end view of the entire customer relationship and interaction, the whole customer experience. This represents a strategic approach to customer relationship management (CRM) that emphasizes all aspects of customer interactions to derive the right level of service, at the right time, and for the right customer by proactively identifying issues and problems before they become major business handicaps.

At the airline, data silos throughout the enterprise were preventing its knowledge workers from accessing data about its customers, especially their travel experiences, and responding to needs before they become service issues. These problems could include lost reservations, delayed flights without notifications, no point-of-contact information in case of problems, inability to speak to a customer representative, and so on, cutting across all divisions of the airline’s operations—the 360-degree view it could not achieve.

By employing the practice of VDD, the airline implemented an integrated platform for intelligence and information sharing that alleviated this problem. An easy-to-use VDD analytical tool enabled more than 20,000 front-line employees and decision makers to access dashboards and visualized data across many disparate sources encompassing the entire customer experience, thus knocking down the data silos. They were able to identify and assess issues and problems by monitoring trend intelligence. They were able to make proposals, adjustments, and changes to benefit their customers’ travel experience. The airline is in the process of developing executive visual dashboards to further enhance their customers’ experience and improve their strategic market position as a leading airline carrier.

Business problems it solves

Data science and machine learning are used across industries and functional areas within businesses. Here are some examples:

• Anomaly detection

  • IoT and engineering

  • Energy: production surveillance, drilling optimization

  • Predictive maintenance

  • Manufacturing: yield optimization

• Financial services

  • Trade surveillance

  • Fraud detection

  • Identity theft

  • Account and transaction anomalies

• Healthcare and pharmaceutical

  • Patient risk assessment: cardiac arrest, sepsis, surgery infection

  • Patient vital signs monitoring

  • Medication tracking

• Customer analytics

  • Customer Relationship Management: churn analysis and prevention

  • Marketing: cross-sell, up-sell

  • Pricing: leakage monitoring, promotional effects tracking, competitive price responses

  • Fulfillment: management and pipeline tracking

  • Competitive monitoring

Figure 1-4 shows an insurance application driven by data science and machine learning under the hood.

Figure 1-4. An application using data science and machine learning to dynamically set prices within the insurance industry

Why you need it: expected business outcomes

We can embed a model to predict a likely outcome or provide an optimized solution to changes in process parameters directly within business processes. A model provides a competitive advantage because it does the following:

• Enhances capabilities

• Speeds decision making

• Processes large amounts of disparate data types

• Generally lowers the costs of operations

• Generates new revenue streams

• Leads to differentiated products and service offerings

This embedding of a predictive model in business processes is the joint goal of data science and machine learning.

Key considerations, requirements, and capabilities

Implementing the data science and machine learning component requires management support and organizational buy-in.

Many organizations struggle to realize the value of data science and machine learning due to many challenges. However, the #1 challenge that organizations face is operationalizing data science and machine learning workflows. All too often, data scientists are focused on the math; however, data science and machine learning is much more than the math. To realize the full value of data science, organizations need to embed data science and machine learning workflows into business processes and have a mechanism to monitor, manage, maintain, and refresh them over time. This includes activities for the following:

• Supporting the end-to-end machine learning process including:

  • Data acquisition and preparation

  • Model build and selection

  • Model deployment

  • Embedding results in business processes

  • Model monitoring, automatic refresh, and governance

• Automation of key process steps

• Collaboration across different project teams

• Flexibility, extensibility, and integration with open source software

Implementing a data science and machine learning function also requires the appropriate level and degree of analytical talent to appropriately use, interpret, and report the results of modeling efforts so that management can make the correct decisions. The talent must be conversant in statistical and machine learning methodologies and programming languages.

Example: University of Iowa Hospitals and Clinics³

Surgical site infections (SSIs) are a major health care issue affecting morbidity and hospital readmissions. SSIs add more than $20,000 per infection to overall health care costs in the United States. The University of Iowa Hospitals and Clinics, a comprehensive academic medical and regional referral center, needed to handle the costs and implications of SSIs but faced an issue common to most large businesses: disparate data sources, an inability of key analytical teams to access and share data to collaborate on major hospital issues, and an inability to quickly analyze the data they did have.

The hospital management team established the objective of having real-time predictions of the risk of infection prior to and during surgery to minimize the chance of SSI. To meet this objective, the hospital wanted a data science platform to streamline its analytical process, from data aggregation and preparation, predictive model development, model and results deployment, to model monitoring.

A data science platform was used to train and deploy a machine learning model (naive Bayes and support vector machines) that would predict a patient’s risk of SSI in real time while they were in the surgical theater. Training data included variables from the Electronic Medical Record, patient history (for example, age, sex, ethnicity), and surgical data (such as surgical apgar score, preoperative hemoglobin, estimated blood loss). If the risk was deemed too high, the surgical team would take action during the surgery and would apply a technique called negative wound therapy that would drastically reduce the risk of an SSI.

The hospital was able to reduce SSI by 58%, increase provider satisfaction, and increase provider engagement in quality improvement efforts. This resulted in a $2.2 million cost reduction for the hospital for every 300 surgeries.

Embedded data exploration and visual analytics

Embedding data exploration and visual analytics into applications and business processes gives users the ability to seek answers to questions on their own without the need to use a separate analytics tool. It gives them an in-app self-service analysis environment for analyzing data and extracting information related to their unique or custom questions.

It is well suited for the following:

• Adding value to applications requiring frequent ad hoc analyses and prompt diverse types of questions from users

• Reducing the time application developers spend fulfilling custom analytics requests

• Empowering decision makers to ask questions and seek more information

Embedded data science and machine learning

Embedding data science and machine learning into general business applications helps organizations realize the value and power of these two tools. Operationalizing them by embedding their algorithms directly into mission-critical business processes reduces costs and increases efficiency through faster on-target decision making. It gives business users the ability to “see into the future,” allowing them to take speedier and well-informed actions, thus giving the business a competitive advantage.

Embedding algorithms into business processes is particularly well suited for the following:

• Systematic business operations involving data for repeatable decisions (for example, credit and loan decisions)

• Augmenting human expertise (such as health care diagnosis)

• Enabling anomalous identification in a mass of complex data (as in fraud detection)

• Identifying and extrapolating trends

Embedded reporting

Embedding reports and dashboards into an application allows users to consume information related to the app and take data-driven action without changing context. Most business users do not want to use an “analytics tool” with yet another interface to learn and another login. They just want easily accessible answers with little time-consuming effort. Instead of offering a standalone report or dashboard, embedded reporting brings those insights into applications already used every day. Figure 1-6 depicts an embedded report.

Figure 1-6. Education application with embedded reporting used by teachers to track performance and deliver tailored coursework to students in grades K–8

Embedded reporting is particularly well suited for the following:

• Presenting information visually and in context to application users

• Providing information to common questions users ask

• Improving customer satisfaction and making a product/service more competitive

• Increasing adoption of BI/data-driven decision making

• Reducing the time application developers spend fulfilling custom report requests

Example: Anadarko⁵

Anadarko, a leading hydrocarbon exploration and production company, was collecting massive amounts of real-time drilling data, often in the millions of data points, from the many drilling rigs it maintains and operates. It needed to improve its operations by lowering operating costs in order to deliver more value to its shareholders. Its Advanced Analytics and Emerging Technology group adopted a streaming analytics platform to process this real-time data. This platform provides the following:

• A flow diagram–type user interface (visual development environment) that is easy for an engineer or a data scientist to use

• Access to analytic engines, providing easy processing of any kind of data

• Capabilities for writing predictive models using open source software, such as R and Python, all in one platform

The result was a faster turnaround time for analysis development and a more optimal use of internal personnel that reduced consulting fees and IT support. This gave the company an advantage against the competition because it could better understand drilling operations and lowered costs and could return more value to its shareholders. Figure 1-7 illustrates this platform.

Figure 1-7. A streaming analytics dashboard

Discover: data exploration and visual analytics

This is an approach that business data analysts use to uncover and investigate hidden but potentially actionable, insightful, and useful information in data. It is a methodology or philosophy for digging into data in order to look for interesting relationships, trends, patterns, and anomalies requiring further exploration and testing.

Key benefit

Provides the ability to visually interact with data in a self-service environment enabling a flexible analytics deployment across the enterprise.

Key consideration

You have business analysts with a good understanding of their domain and are responsible for exploring data and generating insights, but need more powerful tools to explore and discover.

Predict: data science and machine learning

These are jointly used to predict a likely outcome or provide an optimized solution to changes in process parameters that can be embedded in business processes.

Key benefit

Provides a competitive advantage by enhancing capabilities, speeding decision making, processing large amounts of disparate data types, generally lowering costs of operations, and generating new revenue streams, all of which lead to differentiated products and service offerings.

Key consideration

Requires management support and organizational buy-in. Enhanced analytical talent is also necessary.

Present: reporting

This is the practice of collecting, formatting, and distributing rich information in a readily digestible and understandable fashion. It is the final step in an analytical process focused on presenting information to support decision making.

Key benefit

Provides a curated, consumable view of information designed to help users answer a particular question or set of questions.

Key consideration

Reporting tools should work flexibly with data and should support both precision design control of reports and dashboards and methods of distributing them to large audiences.

Analytic enhancer: embedded analytics

This is an effective way to incorporate data analytical methods in business applications and systems to enable business managers and executives gain more detailed and richer actionable, insightful, and useful information when they need/want it and how they want it.

Key benefit

Provides application developers and IT professionals the ability to surface insights in context for their customers and business. It also allows users to delve into their data, ask their own questions, and answer them as they determine is necessary. Most importantly, it allows them to take speedy and timely action because the analytics is embedded in their mission-critical operational systems.

Key consideration

Which components do you want to embed? There are many options, as discussed earlier in “Analytics Enhancers: Extending Analytics with Context and Speed”.

Analytic enhancer: streaming analytics

This is the discipline that applies logic and mathematics to data to provide real-time insights for quickly making better decisions.

Key benefit

It handles real-time and historical data to provide a sense of urgency and call-to-action.

Key consideration

The degree of precision and urgency of your decisions; the mission-critical nature of your processes.

This report has reviewed current trends in data analytics and discussed the options available to you for handling the massive amounts of data you are collecting—often in real time. Your options might seem overwhelming, but as we hope this report has demonstrated, there are ways to reduce the stress involved in handling data and make data analytic decisions that are appropriate for your organization or business and your customers.