Objective

This objective of the document is to crystalize insights acquired through practical experience of delivering enterprise bot solutions that work. These practices enable our team to build robust conversational experiences with relative ease. Designing computer to human conversations is difficult because human-to-human conversations are deceptively easy – which can vary from simple to highly contextual, non-linear, and sometimes even invisible. The main aim is to leverage what we know about human-to-human conversations and use this information to design our user experiences. We will talk about the factors bot designers and developers should be mindful at each stage of a bot project – ideation, training and building, and maintenance. Here’s how you do it:

Ideation

The bot scope

A bot project begins before any intent is configured. The first step for any bot project must be outlining the problem the bot is supposed to solve based on the business needs. This needs to be backed by data to some extent – for example, if you’re building a question answering bot to automate high-volume, low-complexity queries, the bot should aim to answer questions that are being answered by your customer care agents today. If your use-case is complex and you anticipate that the users will need human assistance, the ideation phase is the right time to decide if agent escalation will be provided and through what channels.

Once the key user outcomes are defined and it is deemed that conversational UI is a feasible way to achieve those outcomes, establishing the bot scope and key user intents is as good a place to start as any. It is necessary to map what the users can or cannot do with your bot, while considering both the key user outcomes and technological constraints.

The bot persona

To create an effective conversational UI, you need to have a clear picture of who is communicating. Defining a persona or a conversational partner for your users at an early stage helps you write consistent and engaging conversations. As humans, we have an affinity to project personas when communicating, be it with friends or strangers. Purposefully design a persona and maintain it throughout the conversation, so users do not experience confusing dialogs that feel like they are conversing with multiple personalities. Consistency is key.

If you are unsure as to what persona you should embrace, look at the brand’s core attributes and user demographics. Take the use-case of appointment booking. If you are doing it for a hospital, an apt persona would be one that is sober and straight to the point. On the other hand, the same use-case for a chain of salons allows your bot persona to be quirky, obsessed with fashion trends and not afraid to give unsolicited fashion advice.

Sketching basic flows

Creating a flowchart provides a structure to your chatbot use-case. It determines how various parts of your conversation will connect and how users will interact with your bot. You will need paths clear enough for a first-time user to navigate while being flexible enough to allow shortcuts for advanced users. Make sure your users can switch between various flows and go back and rectify any erroneous information.
This is an essential part of our design process. It is important that the first two steps are implemented before this; you need to establish who’s communicating (chatbot’s persona) and what they’re talking about (bot scope) before planning the way they carry out the task. Now, you must create and build all paths a user could take to reach the outcome. Some users will need more guidance to get to the end than others, while some will try to complete the multi-step flow at one go. You will also need to humor the sly users who will go to any end to ‘break’ your bot and make it fail. The possibilities are endless. The more non-linearity you accommodate for, the more engaging and robust it will be. You will need a diagramming tool such as MS Visio, XMind or so on, which is available online for free.

If you do not want to break the conversation when bot fails to handle the queries or explicitly ask for human agent assistance, the agent handover toggle comes into picture. Enabling the agent handover setting will hand over consumers to the configured customer care platform on invoking this article. When the agent handover toggle is disabled, developers can use the response of this article to specify that no human handover provision is available. In such cases, subsequent consumer messages will be addressed by the bot even after invoking ‘Talk to an agent’ article.

Training and building

Writing dialogs that sound natural

Once you have a structure for your bot, you might start development as quickly as possible. This can be detrimental without writing down the dialogs first.
Write down multiple and separate dialogs that users can potentially go through. Other than the ‘ideal path’ where the users accomplish their goal in the easiest way, you need to write additional paths where users might or might not get to the desired result.
Once you are done writing your dialogs, say them out loud, test them using simulator tools, and make sure they sound conversational.

Intent and entity design

The quality of intent and entity training data is the single biggest factor in determining bot performance. Low quality training data will inadvertently lead to poor bot accuracy in answering user queries.
Make sure that the intents are as distinct enough as possible and do not overlap with each other.
Key considerations for defining training data once you have defined intents that are distinct enough and cover the bot scope:

Training utterance quantity

Training utterances are sentences that determine different ways in which your bot can understand a specific intent.
The number of training utterances required to achieve optimal accuracy depends on factors, such as total number of intents, how distinct these intents are, the NLU engine in use. A good benchmark regardless of these factors is to have at least 20 training utterances per intent. Aim for 50+ if you can. Our experiments and experiences show that with 20 utterances per intent, bot can classify user utterances with ~75% accuracy (given that these 20 utterances match all the other qualitative recommendations listed further).

Training utterance quality

Given how key these are to your bot’s accuracy, it is imperative to have high quality training utterances. If you are automating queries that are currently landing in other customer care channels, the best way to gather quality utterances is to use this live data and use sentences employed by users for these intents. Input from human agents that currently answer these queries can prove invaluable.

If this data is not readily available, put yourself in the users’ shoes and imagine how they would inquire about a particular intent. The practice of writing natural dialogs in previous step comes in handy for this. Getting inputs from different teams within the organization will make this data more comprehensive.

Training utterance variations

Use different variations of user utterances, use sentence which mean the same thing but are constructed in a variety of different ways. Create utterances of varying lengths – short, medium, and long. Change the words by using synonyms for important words. Vary the placement and position of entities in utterances. Do not refrain from including common grammatical mistakes in the training data. Use different wordforms (plurals, different forms of verbs, gerunds, etc.)

It is important to vary the key terminology that defines the intent as well. Using the above recommendations, you might create different utterances such as ‘pay bill’, ‘I want to pay my bill’, ‘make a bill payment’. The core words ‘bill’ and ‘pay’ (in different forms) are still being used everywhere. Be sure to use alternatives like ‘settle my account’, and so on, for these as well. Use of domain specific keywords and regional slangs will also enrich your training utterances.

Training utterance balance

Try and have a similar number of training utterances for each intent to avoid introducing bias towards intents that have more, and against intents that have fewer training utterances.

In bot builder, there is a provision to add an intent and multiple utterances. An intent represents a task or an action that a consumer can perform in each conversation turn. The bot matches the consumer utterance to the bot intent. It is required to have at least two utterances for each intent to train the corpus and fetch an appropriate response. For more information, see Task bots module.

Designing responses

Start and end conversations appropriately

Use your first message to greet the user. Follow it up by conveying your bot’s persona and its capabilities. Aim to make a good first impression and make sure new users understand what your bot does.
Once the intent is fulfilled, give the user a chance to do something else or sketch out an exit experience that allows them to move on with their lives. Make sure that simple requests to end conversation like ‘bye’, ‘no thanks’, so on, are honored.

Take turns and give conversational cues

A good conversational partner knows how to give the right cues. Give the user enough context to respond each time it’s their turn to provide the input. Do not ask ambiguous and open-ended questions. Also, make sure that your bot doesn’t keep sending messages after asking a question. Make sure your bot understands small talk and common-sense intents so that your users have something to talk about even after many interactions.

Leverage the channel

All major digital channels such as Facebook, WhatsApp, Apple Messages for Business, Google RCS come packed with their own versions of pre-built templates that allow you to send structured messages for a richer experience. Use these structured inputs such as Quick Reply buttons, carousels, audio, video, and so on., when possible. This allows you to limit the range of input and avoid some complications of parsing unpredictable input. While doing this, make sure you do not go overboard and respect the chat medium. It would be counter-productive to drop entire pages from your app into a conversation.

Our response designer enables you to configure responses using the GUI based interface. You can configure either of the two responses for an intent or article. The conditional responses and code snippets are two types of responses which contain different response templates. These include – text, carousel, quick reply, image, audio, video, code, file, list picker, time picker, rich link, and media. Code-savvy developers can use python syntax to configure responses for articles or intents. The user-experience is channel-specific for bots on different channels. For more information, refer Response designer module

Be transparent to inspection

Remember that you are spending all this time on chatbot design to help your end-user, not to fool them into believing that they are talking to a human. Pretending to be human while confining to a handful subjects and linear conversation routes will confuse your users and result in awful experience. This does not prohibit you from giving the bot personality if users are aware that they are talking to an automated service.

Prepare to handle errors

Prevent errors by expecting variations while you write the dialogs. Understand input that is phrased in different ways. Reframe questions for users when they give unexpected responses to your prompts or direct them to another question depending on the conversation. Irrespective of the effort that goes into designing, there will come a time where your chatbot won’t be able to recognize user input. When that time comes, ensure you fail gracefully by displaying an appropriate message.

Provide appropriate escalation channels

It is advisable to always have a human fallback option, be it instant transfer to a chat agent, escalation through your CRM platform or providing alternate ways to contact your company. There will always be cases too complicated for your bot to handle and users who would rather talk to a human, irrespective of the complexity of their problem.

After multiple fallback messages, the chat conversation of a consumer can be handed over to the human agent to take it further. This can happen using the allow agent handover and enable handover on fallback toggles in the Bot builder app.

Maintenance

The underlying technologies, such as machine learning and NLP, that power bots and other classification systems are probabilistic in nature and are constantly evolving. This makes bot projects fundamentally iterative. Your job is not done as soon as the bot goes live, it’s just beginning. The long-lasting success of any bot depends on regular, data-driven maintenance.

Different NLP modules such as – common sense, custom named entity recognition, language detection, language translation, sentiment analysis, spell check, chunking, contractions, emotion classifier, lemmatization, named entity recognition, negation tagger, number recognition, numeric range detection, OCR, object detection, POS tagging, parse tree generation, phone number detection, profanity filter, question detection, sentence tokenization, sentiment analysis, text to speech, time & date recognition, units recognition and word tokenization can be added to the bot pipeline to allow the chatbot to understand the consumer message and respond appropriately. For more information, refer NLP module.

This consists of fine-tuning your current intents based on how end-users are asking for them and adding new intents for frequent user queries that are not a part of the current bot scope.

Bot builder curation console was designed with this in mind. Our curation console helps users in optimizing their bot's performance over time through human-in-the-loop learning. It facilitates users in reviewing cases where their bot performance was underwhelming. These can be curated to improve current intents or to create new intents. Incoming messages can end up in Curation based on certain rules and settings that you as a bot developer can control.
Chatbot connected to the knowledge base can store various topics and add topics that are missed at the time of creating a knowledge base for the bot. When a consumer asks a question that a chatbot could not answer, the question can be flagged to add to the curation. This question can either be linked to the related articles in the knowledge base or can be created as a new article or intent.
For more information, see Curation module.

It goes without saying that these best practices will evolve with the underlying technology, NLP capabilities, and the marketplace. Applying these rules methodically will – in most cases – make the design process more efficient and resulting chatbot more effective.