In previous chapters, we explained intents and different ways of classifying intents using natural language techniques. We also discussed the various data sources that are available in designing an enterprise chatbot. There are many chatbot builder platforms and frameworks available in the market that can be used to build chatbots. These frameworks abstract much complex functionality and provide components that are reusable, extendable, and scalable.
Provides better security and control
Data protection from third-party vendors
Minimizes operational cost
In-depth analytics
Flexible design of architecture
Change control management
Interoperability
Easy and quick integration with enterprise-wide available services and frameworks
Integration with messenger platforms
Integration with custom machine learning models
Flexibility to customize with changes in the organization
In this chapter, we will discuss and implement a custom-built chatbot called IRIS (Intent Recognition and Information Service) . We will explain the implementation concepts of the understanding developed in previous chapters. We will talk about designing and implementing state machines, transitions from one state to another in a conversational chatbot, and how they are critical to maintaining the context of user utterances as well as how to make a chatbot mimic human conversation with short-term memory and long-term memory.
In IRIS, the backbone engine of the chatbot is written in Java, and an integration module connects IRIS with different messenger platforms such as Facebook Messenger. The integration module is written in NodeJS, which is discussed in the next chapter.
Introduction to IRIS
We developed IRIS as an open source chatbot framework to provide a novice level of understanding and implementation of a chatbot from scratch. IRIS provides the ability to use our templates of machine learning-based extraction of information from users’ utterances, such as by using named-entity recognition (NER). It provides customized enhancements such as custom intent matching implementation and conversational state management and many other features.
The design is inspired by our collective experience and exploration of how other popular frameworks such as Amazon’s Echo natural language understanding model, Alexa Skills, RASA, Mutters, Dialogflow, and Microsoft Bot Builder are designed and implemented. IRIS derives many methods and implmentation from Mutters, an open source Java-based framework for building bot brains, and reuses some of its design and code concepts to create a simple and modified backend code base. Platforms like Mutters provide a lot of out-of-box features, support, and easy integration. Apart from our custom chatbot framework, we will discuss widely popular platforms and frameworks, and how they work, in the next chapter.
Intents, Slots, and Matchers
A name
Sample utterances
Slots (entities)
A slot matcher
Intent name | Restaurant search |
---|---|
Sample utterances | Looking for restaurants around me Restaurants nearby Best restaurants near me Good continental restaurants nearby Best Chinese restaurants |
Slots/Entity | Cuisine |
Slot matcher | Custom Entity Match Model |
Now, we’ll procedurally go through all the steps involved in creating intent, slot, and matcher classes in Java.
In our new Java project, we will create a package called com.iris.bot.intent in which we will define classes required for intent creation and classification.
Intent Class
Now that we have defined Intent, we need to define the IntentMatcherService class.
IntentMatcherService Class
In the code snippet above, Intent matchedIntent = getIntent(utterance) is the method that provides intent classification.
The getIntent Method of the IntentMatcherService class
There are two essential things to be discussed in the getIntent method , and we cover them in the next sections.
Intent Classification Service
General Query Intent
Most chatbots today are based on general queries and look like an automated Q&A system. The reason for this is that most developers are unsure how to model the chatbot to be conversational. Also, they find it difficult to make the bot interactive.
A general query is never an explicit intent in a chatbot that is conversational and that mimics human conversation modeled as dialogs. Hence, when no intent is matched by the classification engine or if the match probability is not good enough for that utterance, we tend to classify it as general intent. We have seen that this approach is very efficient in practical situations. In another way, if the intent engine is not able to classify the utterance, the utterance could be a generic ask and not aimed for a specific action. We will show later how to use this intent for first looking for an answer in a FAQ repository and then later as a fallback, performing a general search to return a relevant response if possible.
Matched Intent Class
Slot Class
We have so far covered how intent- and slot-related classes can be defined for IRIS. We briefly discussed how IRIS memory is managed by session attributes. Let’s go through this in some more detail.
IRIS Memory
A conversational chatbot needs to hold certain information to be able to closely mimic human-like responses. IRIS is designed to hold information in memory through sessions.
Long- and Short-Term Sessions
Long-term attributes
Short-term attributes
Long-Term Attributes
Certain entities such as name, date of birth, and gender of the user are information that does not change over time. Also, in the real world, we don’t expect our advisors and agents to ask these details every time we interact with them. In the current design of IRIS, what we have demonstrated is that not all attributes are reset after the user session. The long-term attributes that span sessions are supposed to be held in a fast, reliable, and persistent storage databases such as Redis. Redis is an in-memory database. In the code snippet that follows, we show this using a HashMap. Information in HashMaps are stored in JVM when the application is running and get cleared when the application goes down. Hence, even though they are long-term attributes, unless we persist them in a permanent storage like SQL databases, we can’t retrieve them again.
Short-Term Attributes
Unlike name and gender, certain attributes are limited to the scope of the user session. In most cases, the expectation is that the values will vary in each session. An example is a user providing a ZIP code when asking for an insurance agent nearby or providing a face amount for a life insurance eligibility quotation. Moreover, to manage the conversation flow, certain values such as current intent, state, and last question asked are stored as short-term attributes. Short-term attributes reset with each new session or if a session expires.
The Session Class
So far we showed how to create intent and slot classes and the matchers, and we discussed IRIS long-term and short-term memory. We will now discuss an essential concept for chatbot: conversation management. In the next section, we will explain how conversations can be modeled as finite state machines and used in IRIS.
Dialogues as Finite State Machines
Typically, a simple Q&A-based chatbot or a FAQ-based chatbot is not capable of having a conversation. A conversational chatbot should support complex dialog flow between the user and the bot, and we aim to build a chatbot that can mimic human conversation as much as possible. Usually, chatbots are limited to a request-response based flow and are not driven as dialog or conversations.
We’ll explain this with a simple example. In Figure 6-3, we have a finite state representation of switching off and on a light bulb. There are two states: OFF and ON.
States: Different states a bot can be in and transition to.
Initial State: This is the start state when the user first interacts with IRIS.
Transitions: The action(s) that should trigger a possible state change.
Shields: A prerequisite or condition to transition to a target state.
A state machine can be designed in multiple ways. It can be modeled as a graph, the conversation could be modeled as a script, or it could be implemented using a very naive approach like HashMaps and some classes we will discuss next.
We need to create a new package in our project called com.iris.bot.state to contain the base classes for the state machine.
State
The transition from one state to another may sometimes require a validation condition. We’ll explain this with an example. If you are searching for a restaurant of your choice by interacting with a restaurant table booking chatbot, you can ask for cancellation only if you have booked a table at a restaurant. Otherwise, you cannot reach the state of cancellation.
Shields
Transition
State Machine
We are done with defining the base classes and their implementations. Until now whatever we discussed formed the core of the IRIS framework. Now let’s go further with a sample business use case and use the details to create specific intent classes, their slots, different states, and their possible transitions.
Building a Custom Chatbot for an Insurance Use Case
We discussed in Chapter 1 some of the most common applications of a chatbot in the life insurance industry. Now that we have some idea of the IRIS core, let’s dive into building an insurance-focused chatbot using the IRIS framework.
Account balance
Life insurance quotation
Claim status
An advisor
Answers to general enquiries
Market trends
Stock prices
Weather details
The high-level functional architecture is described in Figure 6-4. There are communication client channels such as Facebook Messenger, web chat, and Alexa, via which the users can connect to IRIS. In Figure 6-4, a channel integration module acts as a gateway module. It integrates with services such as Facebook Messenger, receives the request, and delegates the request for IRIS to respond. The response is sent back to Messenger by this module.
AccountBalanceIntent
AskForQuoteIntent
ClaimStatusIntent
ExitIntent
FindAdvisorIntent
GeneralQueryIntent
GetAccTypeIntent
GetClaimIdIntent
MarketTrendIntent
StockPriceIntent
WeatherIntent
Creating the Intents
All the other intent classes are created in the same way with their intent names. Some of these intents will have one or more slots defined as well.
Age (CustomNumericSlot type)
Height (CustomNumericSlot type)
Smoker (BooleanLiteralSlot type)
Weight (CustomNumericSlot type)
Account Type (AccTypeSlot type)
ipin (IPinSlot type)
ClaimStatusIntent and GetClaimIdIntent intents require claimId (AlphaNumericSlot type)
CustomNumericSlot
BooleanLiteralSlot
AccTypeSlot
IPinSlot
In an actual implementation, this type of slot may not be defined but to highlight how a basic authentication can be implemented, we use this entity. We have considered in the example that users will have their own ipin generated in some way and stored in the back end and that it will be a six- digit number. In a real world, a much more complex number and a set of authentication mechanisms will exist such as username, password, and ZIP code.
In the method snippet below, if the value is 123456, only then will the account balance be displayed. Any other number will result in a wrong ipin provided by the user.
Warning
Never implement such a weak authentication system. It will compromise your enterprise security. The purpose here is to only complete the flow of discussion. In no way do we endorse such weak authentication.
AlphaNumericSlot
Now that we have defined all the possible intents that will be classified by our intent classification service, defined slots, and slot type, let’s see what the IRIS configuration looks like. As explained, intents, intent matcher, slot matcher, and different slot and slot types are defined in the IrisConfiguration class.
IrisConfiguration
Adding States
- 1.
Start state
- 2.
Ask for quote state
- 3.
Get quote state
- 4.
Find an advisor state
- 5.
General query state
- 6.
Stock price state
- 7.
Market trend state
- 8.
Get account balance state
- 9.
Get account type state
- 10.
Get weather state
- 11.
Get claim status state
- 12.
Exit state
In getStateMachine method , we define the state classes and Shields.
Shields
As discussed, Shields provide a Boolean condition for transition from one state to another. If all the information required for transitioning to another state is available, Shields returns true.
We implement five shields in our example in the getStateMachine method , each implementing the validate method .
DontHaveAccTypeShield
DontHaveQuoteDetailsShield
HaveAccTypeShield
HaveClaimIdShield
HaveQuoteDetailShield
The execute method of each state
State transitions
Adding Execute Methods
Let’s start by implementing the execute method of each state described in the example.
Exit State
FindAdvisorState
GetAccountBalanceState
GetAccTypeState
GetClaimIdState
AskForQuote State
GetQuote State
Start State
GeneralQuery State
We mentioned that in a chatbot where there are multiple intents such as a user looking for account balance, claim status, weather details, life insurance quote, etc., the general query is not an explicit intent. We classify an utterance into a general query if no other intent matches explicitly.
- 1.
Match if a user utterance is a question that has an answer in our knowledge repository. The knowledge repository is where the most frequently asked questions and their answers are stored. A knowledge repository could also have general user information parsed and stored in a way that can be queried to find a meaningful answer. A knowledge repository could be represented in the form of a graph, RDF semantic web, or implemented using a simple search engine.
- 2.
If there is no matching answer in our knowledge repository, we perform a search on our portal to find any matching result that could be replied to the user. If there is no response from the search service as well, we reply to the user saying we cannot help on this ask because we don’t have much information about it now.
Market trends, stock prices, weather state, and claim status state require integration with third-party data sources or connecting to a database. We will discuss this in the next chapter in detail.
Adding State Transitions
In the getStateMachine method of the IrisConfiguration class, we define the transitions from one state to another. For example, we can transition to any state from a start state, as explained in the following snippet. The first argument of the addTransition method is the intent name, the second argument is the current state, the third argument is the target state, and the fourth argument is an optional shield.
Similarly, we can create transitions for other states.
However, note that there is a difference in state transitions defined by GetAccountBalanceState and FindAdvisorState. You can go to GeneralQueryState from FindAdvisorState but not from GetAccountBalanceState. This is where we define which transitions are possible from each state. In the example here, we don’t want users to be asking general questions after they ask for account balance details.
I am looking for a retirement account balance
Retirement
401k balance
Annuities balance
Want to know 401k account balance
401k
annuities
What’s the weather in Dublin
My claim ID is abc123 can you tell the claim status
Insurance
401k
Retirement funds
Now, contextually, it is difficult to differentiate between a general query search vs. a response to an account type. Here, we can decide that we will not allow a transition to GeneralQueryState.
Another question is how to understand if it is a general query ask. A general query is never an intent. If no other intent, such as asking for weather details, stock price, market trend, claim status, etc., is applicable and the intent classification engine is not able to classify the user utterance into any of these intent categories with high probability, we by default switch it to a general query.
At this stage, we are done with implementing the IrisConfiguration class, and we have defined intents, matchers, slot, slot types, states, different state transitions, and shields in this class.
Managing State
At this point, we have IRIS ready for the insurance industry. However, to make it functional, we expose it as a REST service. We need to create a ConversationRequest, a ConversationResponse, a ConversationService, and a ConversationController.
Exposing a REST Service
ConversationRequest
ConversationResponse
ConversationService
ConversationController
Finally, there’s the controller that exposes ConversationService as a REST API by creating an endpoint /respond. The implementation is straightforward: the controller receives a GET request, it passes to the service, and the service responds with the response message.
Adding a Service Endpoint
Let’s create REST service endpoint using Spring Boot. In Spring’s approach to building RESTful web services, HTTP requests are handled by a controller. These components are easily identified by the @RestController annotation . The @RequestMapping annotation ensures that HTTP requests to /respond are mapped to the getKeywordresults() method.
We create attributes of ConversationRequest based on attributes that are sent by Facebook. Hence in the request structure we have timestamp and seq. However, we do not make use of these two attributes in the demo implementation for intent classification or state transition. Note that these attributes of Messenger webhook events may change with new versions of the Facebook API and can be used in your code depending on your requirements.
Summary
Let’s summarize what we discussed in this chapter. We started with the idea of building a basic chatbot framework and why a custom designed chatbot is a needed for the enterprise. Then we discussed the core components of the framework.
First, we discussed intents, utterances, and slots, and defined a custom intent and slot matcher. We also created the MatchedSlot and MatchedIntent classes.
Then we discussed IRIS memory and how the session can be used to store attributes for the long term and the short term. We discussed the Session and SessionStorage classes.
We then discussed how a conversation can be modeled as a state machine problem. We discussed the different components of a state machine such as states, transitions, shields, and the StateMachine backbone class.
Then we discussed an insurance-focused use case capable of performing certain actions based on different intents and states. We defined various intents, slots, and slot types for the use case. We added these definitions to the configuration class.
We then discussed all the possible states for the use case and explained the execution part of all of these states. As some of the states require a validator before transition, we discussed shields that are required for our example use case. We briefly talked about the general query state and how to leverage an enterprise search in case the utterance is not classified into any of the explicit intents and does not match any document in the knowledge repository.
We then described possible transitions from one state to another depending on the user intent.
We then discussed StateMachineManager, which uses the configuration and performs intent matching before triggering state actions.
Lastly, we discussed how to make IRIS functional. We briefly explained how to expose IRIS as a REST service by the creation of service and controller layers.
In the next chapter, we will discuss the other chatbot frameworks available in the marketplace such as RASA, Google Dialogflow, and Microsoft Bot Framework. These frameworks, unlike our build-from-scratch approach, provide many plug-and-play features and make development faster. However, we recommend that you understand the requirements of your enterprise thorougly before making a choice between the available frameworks.