Skip to main content

Uniphore Customer Portal

Regression Testing

While implementing U-Self Serve, the NLU engine is fine-tuned so that the new utterances are recognized. Whenever the NLU is tweaked, it is necessary to evaluate its accuracy. Manually testing the NLU accuracy is a time consuming process. To save time and effort, the NLU Regression Testing is automated and using the Regression Testing page the NLU accuracy test is performed.

In the Regression Testing page, you can test the NLU accuracy of a VA for a specific language by uploading a csv with new utterances of Intents available in the VA.

Regression_Testing.png

To run a NLU Test:

  1. From X-Console, navigate to U-Self Serve in the left pane and select AI model from the drop down.

  2. From the AI Model, select the Virtual Agent and click the Regression Testing tile.

  3. Select the Language.

  4. Click Utterance Testing button. The Upload Test Data screen pops up.

    Upload_Test_data.png

  5. In the Upload Test Data popup, add the following details:

    • Description: Enter the Description for the Test case.

    • Mode: Select the Mode as NLU. Currently supports NLU mode only.

    • Select File: Click Browse and select the .zip file to be uploaded. Ensure that the file contains user utterances and saved as csv. Compress the csv file into a zip file.

    • Click DOWNLOAD ZIP FILE TEMPLATE to download the csv template.

      • Unzip the downloaded zip file where you can find the RegressionTestSample.csv with sample data.

      • Remove the sample data and add the new utterances along with all relevant information to the csv file and save.

      • Ensure the CSV file contains the data for all headers and remove if any junk data is found in the file.

      • This csv should be uploaded through Upload Test Data screen for NLU Regression Testing.

    Note

    Up to 1000 user utterances can be uploaded in the csv file and tested all at once.

  6. Click RUN TEST button.

    Note

    For a single VA-Language combination, only one user can run the test at a time.

    It is not recommended to run multiple tests by the same user at the same time.

  7. The Success or Failure message for data upload appears on the screen.

  8. Once you run the test, the Description of the Test case, the User name, Submitted Time, Status are displayed.

  9. The Status can be either In progress or Completed. If the Status is Completed, you can download the completed Test result.

  10. Click Refresh icon Refresh_icon.pngto see latest Status of the test.

  11. When you click the Export icon Import_Icon.png, the test result is downloaded in your local drive.

Zip File Template (Sample csv File)
Regression_Testing_-_CSV.png

The csv file should contain the following Information:

  1. S No: Line number

  2. Intent: Name of the Intent available for the selected VA.

  3. Utterance: Utterance to be tested for NLU accuracy.

  4. Mode: Select the Mode as Intent Detection or Slot Filing.

    1. When Intent Detection is selected, the Intent triggered is matched to the expected Intent.

    2. When Slot Filling is selected, the Slot value extracted from user utterance are matched to the expected Slot values.

  5. Entities: Entities available for the selected VA.

  6. Slot Name: Slot name available in the Utterance.

  7. Slot Value: Slot Value available for Slot Name.

Note

If there is no value available for a cell then add NA.

If any Utterance contain more than one Slot, then additional columns for the additional Slot Name and Slot Value to be added in the csv file.

NLU Regression Test Results

Click Export icon Import_Icon.png to download the NLU Regression Test Result in your local drive.

The test result gets downloaded in csv format.

The test result csv file contains the user utterance data that are uploaded and the details related to Intent Result and Slot Result.

  • For the given utterance, if the Intent is recognized, then the Intent Result column is displayed as True else False.

  • For the given utterance, if all expected Slot values are extracted, then the Slot Result column is displayed as True. Even if a single Slot value is incorrect then the Slot result is displayed as False.

In this section: