Testing and Adding AI Phrases
U-Assist provides a built-in facility that enables an Analyst to test text phrases and complete conversation transcripts against the account's current AI Model, and when needed, assign them to a selected Business Process.
Currently, the test also supports the ability to assign test phrase variants to a selected Intent in the Experience.
Periodically testing a Business Process against the Uniphore Natural Language Processing (NLP) engine, and assigning new conversation phrases to your AI Model will greatly enhance its performance and improve its efficiency.
Using the Test AI Model Facility
The Test AI Model facility can parse manually-entered phrases as well as uploaded transcripts of complete conversations.
From the U-Assist menu section in the Admin Console, select Test AI Model. The Test AI Model page is displayed.
From the Business Process dropdown list, select the specific Business Process which will use the new phrases.
From the Input Method dropdown list, select the method you will use to provide the test phrases to the facility:
Text: Enter each test phrase manually. This method is most useful if you have just a single phrase to test, or a short list of phrases.
Transcript: Import all test phrases from a complete conversation transcript file that you upload to the facility.
The fields on the page will change according to the method you choose.
Testing Text Phrases
From the Channel dropdown list, select either Customer or Agent as the mode for AI testing. The Channel tells the Test AI facility who will use a phrase in the conversation. For example, some phrases are spoken by the Agent, while others are spoken by the Customer.
In the Phrase to Test text box, manually enter the AI phrase you want to test.
To the right of the Phrase to Text text box, click Run Test. The new phrase test will begin immediately.
Testing Transcripts
Choose Transcript for the Input Method:
On the right side of the Upload File field, click Browse.
Locate and select the file that contains the transcript you want to test. The selected file will be displayed in the Upload File field.
Note that the transcript file must meet the following conditions:
It must be a text file.
It must use the standard line-by-line conversation format used for all U-Assist transcripts, for example:
Agent: Good morning kai hotels Customer: Hi who do I speak with to book a room Agent: I can do that for you maam I just need the details and you're set Customer: Great but I need to get some prices first Agent: Sure of course
It cannot contain empty lines.
It cannot exceed 60 MB in size.
Click Run Test. The transcript will be processed.
Reviewing and Assigning AI Phrases
After clicking Run Test, the Result section will be displayed, showing parsed information about the processed test phrase(s).
Note
When running multiple tests, it's best practice to wait several seconds between clicks of the Run Test button, to allow sufficient time for sequential server access calls.
In the initial view, the results display how many instances of each Business Process component were found in each test phrase:
 |
An Analyst can then drill down into each of these components to see whether the NLP Engine correctly recognizes and parses the phrase.
For Intents, the Test AI Model facility supports two distinct modes of recognition:
If the test phrase is identical to a phrase that already exists in the AI Model, the parsed Intent displays a Confidence Level of certain.
If the test phrase is semantically similar to a phrase that already exists in the AI Model, the parsed Intent displays a Confidence Level of suggested.
Using this information, an Analyst can optionally assign the test phrase (or an edited variant of that phrase), to the Intent of their choosing.
In the Result section, on the right side of the Intent component, click the Expand > icon. All recognized Intents from the test phrase will be displayed. Note that if the test phrase is not recognized, then the component will show: Intent (0).
If the test phrase is recognized but its Confidence Level is suggested, you can optionally choose to add the phrase to your AI Model.
To add the phrase, on the right side of the current Intent:
Click the Edit
icon. The current Intent parameters will become editable.If desired, type a new variant into the Source Phrase field for this Intent. The variant becomes a new phrase that will be recognized for the selected Intent.
Assign the new phrase as either a Training Phrase or a Negative Example. For more details about assigning this value, click here.
From the For the Intent dropdown list, select the Intent that should be mapped to the new phrase.
Click Add. The new phrase will be shown as Added to the selected Intent.
Note
Only one instance of a phrase can be added to a Business Process. Identical phrases will not be added.
After you add phrases using the Test AI Model facility, you must also Train the related Experience so that the AI Model will recognize the newly added data. For more details about the training processes, click here.
When a transcript is processed, the Result section of the Test AI Model facility displays each conversation phrase sequentially as separate turns on the left side of the screen, with each of the related Business Process component instances on the right side:
In the Result section, on the right side of the Intent component, click the Expand > icon. All recognized Intents from the test phrase will be displayed. Note that if the test phrase is not recognized, then the component will show: Intent (0)
If the test phrase is recognized but its Confidence Level is suggested, you can optionally choose to add the phrase to your AI Model. Follow the instructions above for each phrase you want to add.
After you add phrases using the Test AI Model facility, you must also Train the related Experience so that the AI Model will recognize the newly added data. For more details about the training processes, click here.