Enabling the new AI (NLU v2) on the Certainly Platform

 

As of July 7th 2021, (see PBI here), the Certainly Platform supports switching chatbots to a brand new NLU service*.

By switching to the new NLU service, bot builders can access NLU predictions such as languages, sentiments, entities(both default and custom ) and of course, intents, directly within the module actions, without the need of employing dedicated Webhooks.

The metadata extracted from the NLU service is available after every visitor reply, under the metadata object nlu.

*The new NLU service is exposed via REST APIs as documented here: Certainly NLU v2 REST API

Included in this article:

Switching chatbots from NLU v1 to NLU v2

(1) If your chatbot is still running on NLU v1, then the following message should be visible in the chatbot's AI dashboard

mceclip0.png

 

(2) Navigate to the chatbot on Django, tick Nlu platform two and save your changes.

mceclip1.gif

 

(3) The chatbot's AI Manager should now display the following banner:
mceclip2.png

 

(4) From now on, new training requests will go through the new NLU service. Therefore, go ahead and train the AI gain, so that you can benefit from the new NLU service.

 

(5) Every time the visitor sends a message, the NLU v2 prediction based on the message will populate the  metadata point nlu. The structure of the nlu datapoint is the same as the response of the /test REST API endpoint from the NLU v2. See the example below for reference:

mceclip0.png

 

The bot will populate cvar with the whole value of the nlu metadata object.

mceclip1.png

 

Example value for the nlu metadata object:

{  "text": "string",  "prediction": {  "languages": [       {
 "name": "string",  "confidence": 0,  "source": "string"       }
    ],
 "sentiments": [       {
 "value": 0,  "source": "string"       }
    ],
 "entities": [       {
 "start": 0,  "end": 0,  "value": "string",  "parsed_value": "string",  "entity": "string",  "role": "string",  "source": "string",  "entity_confidence": 0,  "role_confidence": 0,  "meta": {}       }
    ],
 "intents": [       {
 "domain": "Unparented",  "name": "string",  "confidence": 0,  "source": ""       }
    ],
 "composed_entities": [       {
 "name": "string",  "text": "string",  "entities": [           {
 "start": 0,  "end": 0,  "value": "string",  "parsed_value": "string",  "entity": "string",  "role": "string",  "source": "string",  "entity_confidence": 0,  "role_confidence": 0,  "meta": {}           }
        ]
      }
    ]
  }
}

 

Supported prebuilt entities

The new NLU service supports the following prebuilt entities:

Country Information

Entity LabelEntity valueEntity String

Country

US, the Netherlands

entity/common/country

Postal Code

1050, 28806

entity/common/postcode

Temporal Expressions

Entity LabelEntity valueEntity String

Time

Tomorrow, 1st of April, next weekend, this Sunday

entity/common/time

Duration

4 days, 2 hours and 3 minutes

entity/common/time/duration

Holiday

Christmas, New Years Eve, May fourth

entity/common/holiday

Measurements

Entity LabelEntity valueEntity String

Number

Seventeen, 1.5, 0,7, 212

entity/common/number

Ordinal

3rd, fourth, eighteenth

entity/common/number/ordinal

Volume

300 ml, 2 liters, one gallon

entity/common/number/volume

Distance

8 miles, 4 feet, 42,195 km

entity/common/number/distance

Temperature

80 °F, 72° Fahrenheit, thirty three Celsius, 90 degrees

entity/common/temperature

Amount of money

20 dollars, 5 bucks, $12

entity/common/amount_of_money

Personal Information

Entity LabelEntity valueEntity String

First name

Susan, Casper, Joana

entity/common/first_name

Email

contact@certainly.io

entity/common/email

Phone number

415-123-3444

entity/common/phone_number

URL

https://www.certainly.io/

entity/common/url

Credit card

DinersClub, MasterCard, Visa

entity/common/credit_card

Browser

Mozilla firefox, Edge, Chrome

entity/common/browser

Shipping

Entity LabelEntity valueEntity String

Shipping provider

DHL, Fedex, US postal service

entity/common/shipping_provider

Fashion: Sizing

Entity LabelEntity valueEntity String

Size

XS, medium, 42, 104

entity/fashion/size

Size jeans

28/28, 32*30, 34l

entity/fashion/size/jeans

Size shoes*

(Coming soon)

entity/fashion/size/shoes

Length

normal, short, thigh length, 3/4 length

entity/fashion/size/length

Fit

oversized, relaxed, slim

entity/fashion/size/fit

*Coming soon

Fashion: Clothing

Entity LabelEntity valueEntity String

Category

dress, beachwear, suit, belt

entity/fashion/category

Bag

shoulder bag, clutch, computer sleeve

entity/fashion/bag

Dress

Ballgown, maxi dress, cocktail dress

Entity/fashion/dress

Hat

cap, fedora, tam o' shanter

entity/fashion/hat

Outerwear

wool coat, leather jacket, windbreaker

entity/fashion/outerwear

Knitwear

hoodie, cashmere sweater, pullover

entity/fashion/knitwear

Pants

chinos, jeans, shorts, jumpsuit

entity/fashion/pants

Shirt

blouse, T-shirt, tunic

entity/fashion/shirt

Shoes

dress shoes, heels, boots, mule sandals

entity/fashion/shoes

Swimwear

bathing shorts, tankini, swimsuit

entity/fashion/swimwear

Fashion: Attributes

Entity LabelEntity valueEntity String

Fabric

cotton, jersey, bamboo

entity/fashion/fabric

Occasion

casual, festive, cocktail party

entity/fashion/occasion

Color

blue, red, green

entity/fashion/color

Use case example

This demo chatbot is running NLU v2 and is trained to understand two test intents and three custom entities: item, person, and orderNumber.

 

Extracting default entities

Extracting a default entity (email) from a visitor's message is done by accessing the desired Entity Stringin the nlu metadata object.

In the following example, Module Connection #1 attempts at extracting default entity email from a visitor's message, and uses it to populate cvar :

mceclip0.png

Jinja reference snippet:


 

This results in the following during conversation time:

mceclip4.png

 

Extracting custom entities*

*Important Requires training the AI model with custom entities and configuring the bot to use the new AI model. Any retraining of the AI inside the platform will result in the bot adopting a new AI model, and the custom entities being lost.

Extracting two custom entities (item and person) from a visitor's message:

mceclip5.png

Jinja reference snippet:


This results in the following during conversation time:

mceclip7.png