Chatbots are complete solutions and are created as an independent layer in any solution. The senior management also looks at chatbot functionalities and ROI as an independent entity. The focus on conversational technologies further demands a holistic view on chatbots from solution and business returns perspectives. In previous chapters, we demystified the essentials of developing a chatbot for a closed domain. In this chapter, we will focus on how to build solutions using the best available resources for a closed domain use case. The chapter will also cover a thought process on how to measure success for a chatbot implementation and managing the risks associated with chatbots.
Business Considerations
Any business will ask the question, “What business value does a chatbot add?” This question is to be answered with objectivity and time targets. Technological advancements may allow us to implement advanced chatbots and other solutions, but how they add value to the business is a very subjective call. The business needs to evaluate all factors to ascertain how a chatbot is good for their business.
Chatbots vs. Apps
From technological point of view, the business must tackle an important question, specifically relevant to closed domain chatbots: whether to go for an app or a chatbot. In terms of functional features, both can provide the same information for a given feature set. The key differential happens to be chatbots being conversational in nature, while apps are self-service applications.
Chatbots are preferred to get quick answers for questions and 24-hour access.
Apps are preferred for ease of use and convenience.
Quick answers to simple questions
Getting 24-hour service
Convenience
Quick answers to complex questions
Ease of communication
Ability to easily register a complaint
Getting detailed/expert answers
A good customer experience
Friendliness and approachability
Having a complaint resolved quickly
Growth of Messenger Applications
Another factor driving the need for chatbots is the increasing usage of messenger applications and a stable growth rate of having a mobile-focused approach by companies. The customer is now connected 24 hours to the Internet through their mobile handsets and wants to access services via easy-to-use interfaces.
In early 2011, messenger applications started coming up and had good adoption rates as mobile devices, internet connectivity, and cloud computing also picked up at same time ( www.businessinsider.com/the-messaging-app-report-2015-11 ). Sometime in early 2015 we got to the point where messenger applications, specifically WhatsApp and Facebook Messenger, were at meteoric adoption rates and equalled the activity on social networks. As the trend suggests, people are more active on messenger applications compared to social media networks.
This trend points out the shift in consumer behaviors where chat is a preferred mode of communication. And this implies that businesses need to enable this channel of communication with customers as well using either chatbots or human chats.
Direct Contact vs. Chat
The increasing use of messenger apps has shifted the way customers want to interact with businesses. In the early days of messenger apps, the studies showed an increasing preference for contact via chats.
The survey by BI Intelligence as summarized by Chatbots Magazine ( https://chatbotsmagazine.com/chatbot-report-2018-global-trends-and-analysis-4d8bbe4d924b ) shows that as early as 2016, the mature market customers adopted chats very quickly. The consumers believed they could solve their issues faster over chat than calling a customer care representative. They also felt more comfortable with chats because they could keep them as a record for follow-ups, unlike phone conversations.
Business Benefits of Chatbots
Considering essential business aspects allows a company to decide whether it wants to go ahead and build a chatbot or improve current apps/channels. If the company decides to go ahead with a chatbot solution, it needs to understand the key value creation by chatbots. Chatbots are now accepted by users with different percentages in different industries, with different adoption rates.
In the aforementioned Chatbots Magazine questionnaire, which asked how comfortable you are with being assisted by an AI-based chatbot for business communication, the response showed that people are most ready for such conversations for online retail, generic healthcare queries, and telecommunications.
Further, in the 2018 State of Chatbot Report, the top reasons for not preferring chatbots are the need for assistance from a real person, less awareness of chatbots, and possible blocks due to lack of accessibility of channels (i.e., not having a Facebook account or access to a smartphone).
The studies indicate that there is a value in adding this channel if the benefits for the company are high and your customer is comfortable being assisted by AI-based chatbots. The two topmost value creations from chatbots are discussed in following sections.
Cost Savings
Undoubtedly the most important benefit for the company is cost savings on customer service. The cost savings makes the best case for bringing chatbots into the service delivery channel of company. While there are lot of other strategic and growth benefits, the cost needs to justify all the efforts and resources used for chatbot development and maintenance.
A BI Intelligence study ( www.businessinsider.com/intelligence/research-store?&vertical=mobile#!/The-Messaging-Apps-Report/p/56901061 ) shows potential annual savings on salaries by augmenting the chatbots across various business functions. The highest expenditure area for salaries is the customer service representatives, where the savings are also highest. The cost reduction is clearly visible and attributable to the use of chatbots across the functions in insurance sales, reporting, sales, and customer service.
Customer Experience
High brand value and recall
High lifetime values (improved engagement)
Brand differentiator from competitors
There are many other derived factors due to a happy customer experience. A loyal customer base is a recipe for long-term success.
24-hour service
Getting an instant response
Answers to simple questions
Easy communication
Complaints resolved quickly
A good customer experience
Detailed/expert answers
Answers to complex questions
Friendliness and approachability
The variety of new features attracts customers and creates unique value for companies.
Success Metrics
Success metrics are important to define at the start of any chatbot development. The metrics work as a compass to direct the solution and the intended benefits of the chatbot. While there are success metrics related to the accuracy of the NLP engine, the intent classifiers, and other technical aspects, in this section we will only talk about success metrics from a business perspective.
The success metrics need to be manageable and measurable with a simple explanation to the business. We will discuss a few metrics that can be used to track and manage the success of chatbots. The metrics focus on success when you compare a chatbot interaction with a human interaction.
Customer Satisfaction Index
The Customer Satisfaction Index (CSI) measures a customer service representative’s quality of interaction by following up with the customer with a small survey and capturing their experience of the interaction. CSI is one of the most impactful metrics to monitor because it provides not only the satisfaction scale but also the areas of improvement.
Completion Rate
The Completion Rate (CR) is defined as the proportion of interactions with a chatbot that ended as the solution resolved for the customer. This metric tells us how many times the chatbot can complete a conversation and deliver the required responses to the user. A higher completion rate indicates a more efficient chatbot service.
Bounce Rate
Bounce rate (BR) can be defined as how many users move away from chat after typing in one or two inputs to the chatbot. A high bounce rate means the chatbot is not successful in engaging the user and this must reflect in some of the customer feedback.
Along with bounce rate, we also measure the reuse rate (RR), which refers to how many customers come back and use the chatbot again. BR is a perfect metric to identify those people who more tend to use chatbots and target similar customer segments.
Managing Risks in Chatbots Service
New technology channels do bring risks. Customers and companies need to understand the risks involved with using chatbots for any transaction or information exchanged through chatbots. The risk is to be understood, communicated, and mitigated before general customers are allowed to use the chatbot services.
Third-Party Channels
Banks and other financial institutions are very much aware about opening a new channel for users to access financial services. While it adds convenience to the customers, it brings some risks as well. Technology is growing way faster than the risk frameworks we have. By the time a general user is able to figure out the risk with usage or best practices for using chatbots, they may already face a security breach.
Top security risks arise from the communication channels for chatbots because they are external to the bank’s security control. For example, a customer interacting through Facebook Messenger is interacting with the bank systems using the Facebook platform, which may have vulnerabilities and is not designed for banking operations, just generic chats among people.
In a 2018 survey undertaken by Synopsys, 36% of respondents indicated that customer-facing web applications remain the top security risk to businesses in the Asia-Pacific. September 2018’s admission by Facebook that a security breach had affected more than 50 million accounts came as a timely reminder that even tech giants aren’t spared. (Source: finews.asia).
These cases require financial institutions to limit functionalities through public channels for chatbot messengers. Developing an end-to-end chatbot experience can reduce this risk as well, but adoption remains a challenge.
Impersonation
Another very prominent risk arises from impersonation. Impersonation can result in similar looking chatbots, or humans having conversations using fake windows, hacking social media, and other sources of impersonation. The banks already face a lot of fraud due to criminal ingenuity from fraudsters and spend millions in education for phishing, vishing, and other impersonation attacks.
Two-factor authentications are one possible way to reduce the impersonation attacks by having two-step verification from two different sources. In most cases, hackers are not able to crack both factors of authentication and are less likely to be successful in fraudulent transactions.
Personal Information
Personal information revealed through the chatbot channels is a challenge for banks to manage. It is challenging to control users who may accidentally enter their personal information to get access to a service. As the chatbots are driven by natural language, the chances of revealing personal details are high.
The chatbots need to make sure that they use as little information as possible. It’s better to create a PIN with adequate access control so that the user never needs to disclose personal information; they can just use the PIN. Educating the users is an essential step to make sure the user is aware and alert for any fraud.
Confirmation Check
Confirmation is the most impactful and sometimes last resort to make sure transactions done via chatbots are legitimate. Fraudulent or mistaken transactions are possible using chatbots. As new technology comes into user service, it takes time for the users to understand the right use of the service, and in this process, they might do some illegal transactions as well.
For any transaction that seems to be an anomaly or unexpected, it’s always good to call up the customer and ask for confirmation before processing it. This check saves the user and bank from fraud.
Generic Solution Architecture for Private Chatbots
Solution architecture is a practice of defining and describing an architecture of a system delivered in context of a specific solution and as such it may encompass a description of an entire system or only its specific parts. Definition of a solution architecture is typically led by a solution architect.
Source: Wikipedia
Solution architecture of the 24x7 Insurance Agent
The architecture captures at a high level how the solution will work. This is not the same as technical architecture, which explains the specific components and their configurations to make the solution work. The precise technical architecture is built as per requirements and is out of scope for the book to cover. The following subsections expand the solution architecture with other vital information to explain it.
Workflow Description
- 1)
Conversation interface: We will develop the interface from scratch; it does not depend upon third-party interfaces (e.g., Skype, Telegram, etc.). This will help us create customized interfaces for the organization’s need and extend other features as required. The interface can be an entirely new mobile app for the iOS, Android, and Windows platforms.
- 2)
MFA and Active Directory: The authentication system will be built at the back end to authenticate devices (MS Intune), users (Active Directory), and applications (by PIN). We will make it on a node environment to allow integration to other identity management services as well. In the natural form, we will only have PIN verification to access the application.
- 3)
NLP engine: The NLP engine will be built to accept text inputs from a queue and extract intent and apply context to the incoming query. Once the question is broken into the required components, it’s sent to the bot logic.
- 4)Bot logic: This is the core handler of the incoming request. It will have two core inputs before processing the request. The bot logic does not call information services until it has satisfied the process set as per the below two methods. If there isn’t enough information for the bot logic to reply, it’ll ask the user for more information.
- a)
Policy interaction flows: These flows are bank expert-designed workflows for the incoming request (e.g., if someone asks for an update of address, what are the essential steps for a reply?) The steps will make sure the user complies with the steps to get an answer. This ensures that all policies, statutory or internal, are followed by the bot, just like an informal HR. Also, policies and FAQ can be defined here.
- b)
Machine learning: The request that requires a machine learning algorithms to improvise the output is requested from here (e.g., can I request a statement from March 15 to March 18?) This needs a machine to apply appropriate logic to extract dates, an employee id, and an existing account balance to create the right query to the information system. Further, sophisticated features like the mood of the employee, the urgency of the request, and the sentiment of the question will be built here.
- a)
- 5)Information service: This is the place where real-time information is fetched for the employee request. This service handles all appeals and prepares responses AND can also send a request for actions as well. The information service will talk to three core data services:
- a)
Data lake/databases: If some data needs to be fetched from some database or data lake of corporate.
- b)
Third-party APIs/ODBCs: Interacts with HR systems via ODBC/APIs or some other method that exposes itself with REST APIs.
- c)
Human HR: If there’s a low score of confidence in the reply, it will transfer the request to HR for a response through the chat interface.
- a)
- 6)
Actions: The information service will also route the requests asking for some work to the actions item management queue (e.g., a request for leave for tomorrow). This request requires an update in PeopleSoft or the HR system. All such change requests will be routed to an HR approval (can be direct as well); once approved, either by real HR or by a policy-based rule, it will be sent to an action queue. HR can support broadcast here, and they can be delivered by an actions workflow.
- 7)
Updates in real systems: For authorized and approved requests, we update them directly in the HR system and trigger a notification to the chatbot user and also trigger emails and other built-in process flows of the banking system.
Key Features
Built for you: We will make the bot for specific needs, not fitting those needs to existing bot frameworks.
Data privacy by design: The bot is developed with data privacy by design. It will be fully compliant with local laws and internal laws.
Developed with a microservices architecture: The entire application will be based on principles of microservices and hence will allow future-proof design and also advanced application development on top of the framework.
Options for deployment: We can choose which components we want to deploy on-premise or on the public cloud. Based on needs, we can create a deployment plan.
Extensible: We can integrate as many APIs or AI/Ml features as deemed fit for use. All the new future changes can be consumed as APIs in the framework.
Technology Stack
Core engine: Java and JavaScript
Backend server: NodeJS and other JavaScript
Front-end server: Mobile apps based on native frameworks
Log management: Cloud store of a small Hadoop cluster. Also, these stored conversational logs provide data for AI/ML model training.
Visualization: Can be custom created using D3; if self-service is required, then Tableau/PowerBI integration with the logs.
Search: Elastic Search to search the conversation logs.
Maintenance
The technology: Application uptime will be maintained with the help of on-premise engineers and an on-call channel for all queries.
The AI/ML brain: This will be done offsite by a team of data scientists and updates will be pushed to the systems when ready.
Summary
The chapter described the business considerations for a chatbot solution and listed the benefits of the chatbot. The market growth of messengers is a good sign, indicating that users are adapting to messengers, so a service chatbot can add to their experience and reduce the cost for the company. Third-party studies show the impact chatbots are creating in businesses and how they are bringing digital channels close to natural language. The next section talked about the success metrics that must be defined to manager and improve the chatbot. The essential metrics include the Customer Satisfaction Index (CSI) and Conversion Index (CI).
Further, the chapter also discussed the potential risks of chatbots and how to manage them. The most prominent risks are impersonation and hacking of credentials. Both of these risks require education and secure authentication systems. In the end, we show a reference solution to develop a chatbot. The architecture, workflow, technology stack, and maintenance notes provide enough information to build your chatbot solution as per certain needs. In the following chapters, you will learn about critical areas of natural language sciences, including understanding (NLU), processing (NLP), and generation (NLG). And then we will discuss the implementation of features using open source technology and in-house-developed frameworks.