Implementing AI and using the AI Manager

Want to make your bot smarter with AI? Great! You’re in the right place. There are two main steps to using AI in your bot: first, selecting the data you want to train your bot on, and second, establishing Connections based on that data.

To help you get started, we'll provide you with some background information and an overview of the AI Manager, which can be found in the right sidebar menu. In this article, we will address:

Terms used in the AI Manager

First, we'll go over a few important terms. This is critical to gaining a solid understanding of our AI and using the AI Manager.

How to prepare for using the AI Manager

Before working in the AI Manager, it's important to have an overview of what you want to focus on. That's why conducting data analysis can be a valuable step, as you'll quickly get a sense of what topics and questions your users often bring up.

In this section, we'll explore:

Data analysis

The best way to get started is to perform an analysis of your pre-existing data. Traditional chat services such as Facebook Messenger, Zendesk, Service Now, LiveGuide, Dixa, and SnapEngage store logs of conversations. If you already have a bot that is live, then you can also use data from Message Reports.

By analyzing these logs, you will be able to see user tendencies. This information will allow you to determine a structure for your AI implementation. The logs are especially valuable if they are categorized by user intent. Logs from some chat services, such as Zendesk, often include such categorization.

Reviewing this “labeled data” can help you to decide what to focus on and improve the AI's initial performance. This includes:

  • Concluding which Intents are most frequent and important
  • Deciding how Intents should be separated
  • Determining which Intents can be answered automatically, and which require human takeover
  • Gaining insights into which Domain-specific phrases are important for triggering certain Intents

For Zendesk users, macros are often a good indicator of frequently used Intents that can be answered in a standardized manner. We recommend extracting statistics about the most frequently used shared macros.

Some customer support departments might also have libraries of personal macros. In some cases, tags are used to indicate the Intent of tickets. If tags are used consistently, they carry important information that can be used to classify the sentences.

If you're unsure how to export data, feel free to reach out to us at support@certainly.io!

Defining Intents

From the data analysis, you will be able to define your list of Intents. We recommend that you:

  • try to find questions that can be covered by the same Intent;
  • attempt to divide your Intents into what, where, why, and how questions to get an overview of the different query types;
  • define between 10 and 100 Intents to get an overview of areas you should focus on (note that you must add at least two Intents, as we don't allow the AI to operate with fewer); and
  • give your Intents short, easily interpretable names.

If you need more than 100 Intents, you should consider either merging some Intents together or narrowing the scope of what your chatbot addresses. For example, questions about if shipping is free or how much shipping costs should be covered by one Intent, as these both concern questions about shipping price.

You can also limit the use case for the bot. Your chatbot is not an actual human and should only cover a narrow topic. It might be preferable to create multiple chatbots for different Domains, or you could leave out some Intents entirely.

Defining a good set of Intents is important for the AI engine, and we recommend that you review your chosen list of Intents a few times before moving on to the next step.

Writing Example Sentences

Once you've decided on a list of Intents, you should provide the bot with no fewer than 30 Example Sentences for each Intent. It is important to use as much language variation as possible when providing the bot with Example Sentences.

For instance, the terms "Getting the goods delivered", "freight", and "shipping" are all different ways of saying the same thing. By covering all of the different possible expressions, the AI engine will perform much better.

How to use the AI Manager

Now that you have all of your data prepared, you can start working in the AI Manager. When you first open this tool, you will go through a short tutorial. Then you will be asked to choose which language you want to use your bot in.

After selecting this, you will be taken to the AI Manager:

Overview-of-NLU-dashboard.png

Below, we'll cover:

tip
You can revisit the AI Manager tutorial by clicking the question mark in the upper right corner.

Using public Domains

In the AI Manager, you'll notice there are already quite a few folders with stars in this view. These are public Domains.

Remember, a Domain is made up of Intents, and it should represent the overall theme of those Intents. The public Domains have been created by Certainly and contain prebuilt Intents and Example Sentences which can be used as a foundation for your work. We recommend using the public Domains and Intents as much as possible, so you don’t have to start from scratch.

You can always add new Intents to the prebuilt Domains. Similarly, you can add Example Sentences on top of these Intents. All you have to do is hover your mouse to the left of an existing Intent or Example Sentence, and the option to add a new Intent or Example Sentence will appear. There is more information on this below.

Adding a Domain

To add a Domain, press “Add Domain" at the bottom of the AI Manager. Any Domains you add within your company will be available to all of your authorized users, and these Domains will appear in the AI Manager for all of your company's bots.

You'll then be taken to an overview where you'll select a language for your Domain at the top left. Press the box that says, “Start from scratch”. A new folder will be added to the list of Domains in your AI Manager. This is your new Domain. You can name that folder by double-clicking on the text. Please remember to create easily identifiable names for your Domains.

You can also use the "Import Domains" function. When you select this option, it will provide information and a visual example of how to format your data in a .xlsx / .xls file in order to upload it to the AI Manager. After clicking "Import", these Domains will show up on the overview screen. To add one of these new Domains to your bot, simply click on the Domain.

It will then be available in the folders view of the AI Manager:

Domains_Overview.jpg

Updating Domains

If new content has been added to a Domain, you'll see "Update*" at the top of that Domain folder in the AI Manager. This is the case for updates made both by your own organization and by Certainly offering new prebuilt Domains or adding content to existing ones. To include the new content in the AI of your bot, just click on the Domain. The latest content will be added.

This works across bots as well. For example, if you are using the same Domain in two bots, and you add new content to that Domain in Bot A, you can easily include that content in Bot B with just one click in Bot B's AI Manager.

Creating Intents

After you have created your Domain(s), it will be possible to add Intents to the Domain. You must add at least two Intents for the AI to work. If you hover your mouse over the Domain folder, you will see a small plus sign. Click on this, and an Intent will be added. You can name and rename these Intents as you wish.

Hover-intent-1st.png

Creating Example Sentences

For each Intent, you will have to provide a number of Example Sentences. To create an Example Sentence, you will need to hover over your Intent, and a small plus sign will appear. Click this and insert your Example Sentence. Please keep in mind that you should at least have 30 Example Sentences for each Intent.

Sentence-hover-2nd.png

Testing your AI data

After providing the bot with an initial list of Example Sentences, it's time to run the first user test of the bot. On the right side of the AI Manager, you'll see a test area. This area will appear inactive if you haven’t activated the state of any Domains.

To activate a state of a Domain, you have to turn it on by clicking the toggle button under the "State" column. When a Domain has already been trained, its state is marked in purple, and if it has not been tested yet, its state is marked in yellow.

Testing_Domains.jpg

Once you have activated and trained a few Domains, you are now ready to test your bot! Please see the following image for an overview of the AI testing process:

Testing_Domains_2.jpg

We recommend that you run 10 independent tests with 10 colleagues who have not been involved in the development of the bot. Ask them to write a sentence that has something to do with your activated Domains and Intents. The AI Manager will then calculate the likelihood of the question being an expression of your activated Intents.

The results are divided by a confidence threshold. Results below 0.5 won't trigger any Connections, but this information can help you improve your AI. The raw data can be downloaded as a .txt file using the button that appears above the results list.

You can see the download button and threshold divider highlighted in the following image:

aitesterthreshold.jpg

For each test, pay attention to if:

  • there are any Intents that you've failed to cover with your initial Intent definitions, and/or
  • there are any user expressions that the bot fails to recognize.

When your initial tests are over, add any missing Intents and Example Sentences to your data, then retrain your bot. If you find the performance of the AI system unsatisfactory, repeat this step until you reach the desired result. For complex use cases, the AI system might require hundreds of Example Sentences for each Intent to reach a satisfactory performance.

Keep in mind that the more (and more specific) data you can provide your bot, the better!

Inserting your Intents in Connections

To be able to use your work from the AI Manager, you will have to insert the Intents into Connections. The Intents can be inserted in both local Module Connections and Global Connections. Please be aware that prior to inserting the Intents, you will have to activate their state and train your bot in the AI Manager.

When adding Intents in a Connection, you'll first have to select the “AI Understands” condition, after which you'll see a drop-down list of the Intents activated in your bot. Select one of these Intents, then choose which Module to recognize the Intent in.

Next, you'll need to choose a confidence level of the Intent. This should be based on the number of sentences you've added to the Intent and also the results you've gotten from training your bot.

You also have the option of enabling the "Only validates if it has the Highest AI Confidence" function to ensure that the Intent is only recognized if the AI has the highest confidence that this is the correct Intent. That is, if the AI has a higher confidence that another Intent than the one selected is the right one, then it will not match the connection at all.

Lastly, you'll choose an action of the "AI understands" condition. Below, you can see how to insert your Intents in Connections:

AI_Connection.jpg