[VA] Virtual Agent and [NLU] Natural Language Understanding Resources Page including How To videos on YouTube and ServiceNow Community Forum containing blogs and articles to assist with NLU implementation.

Summary

Although our documentation on Natural Language Understanding (NLU) covers all the capabilities and features of this product, implementing NLU can present some challenges, hence we have released additional resources to assist our customers.

ServiceNow Community:

The ServiceNow Virtual Agent and Natural Language Understanding (NLU) Community Forum has blogs and articles, which contain additional NLU implementation resources, as well as information on what is new with each version. Please subscribe to this forum to receive updates in this forum.

Documentation:

Note: Since the release of LLM topic discovery in Virtual Agent in Vancouver, which no longer requires any NLU Models and is considerably easier to manage, we are focusing our efforts on the LLM topic discovery, and with Washington released Migrate NLU topics to LLM topics, to automate the migration from NLU to LLM. Therefore, there will be no further NLU releases other than fixes, as we still support NLU.

YouTube:

We also have the Virtual Agent Academy on YouTube with How To videos on how to implement Virtual Agent and NLU.

Community Articles:

Knowledge Articles:

KB0827206: [NLU/VA] Too many utterances in your NLU Model will cause a "Read timed out" and it is unable to train the model. Testing an utterance on an NLU Model that has more than 3000 utterances can also cause a "Read timed out"
KB0858445: [NLU] In Paris or earlier versions, when upgrading an instance to a newer patch level, when you train the NLU model after the upgrade, the Prediction Response scores will change, as it using a newer version of the NLU Service that was rolled out.
KB0862438: [NLU] How to determine the version of the NLU Platform used to train your NLU Model and debugging NLU predictions on a single NLU Model using the Test button in the NLU Workbench.
KB0864097: Best Practice: Custom NLU Model development using the out-of-box Scoped Applications for NLU Models as templates
KB0951738: Infrequent Natural Language Understanding (NLU)/Virtual Agent (VA) performance issues, where an utterance can take up to 50 secs to process and it eventually returns with no NLU intents.
KB0952640: NLU Model training creates a new record in table [ml_solution] for the trained NLU Model solution throwing an error and unable to publish or test
KB0960761: [NLU] Open-ended entity suggestion feature will throw a "Training Error" and provides a suggestion to change it, if the NLU Model contains an incorrectly configured open-ended entity.
KB0963813: When creating an open-ended entity, after accepting the default entity, it fails to train the NLU Model and throws an error, as it still contains the default entity in the NLU Model JSON and this entity cannot be removed in the NLU Workbench
KB0966069: [NLU] Asynchronous NLU training can take much longer than normal and at certain times not complete, such as Table Vocabulary Sources are "Syncing" for a very long time and NLU Batch Tests that are not completing in the expected time
KB0994284: [NLU Workbench - Advanced Features] NLU Model Performance Clustering analysis for Unsupported utterances never completes and it eventually gets timed out and no clustering report is displayed.
KB0995575: [VA/NLU] Vocabulary (Dictionary) in the utterance is not recognized and it returns an incorrect intent and the wrong topic is displayed in Virtual Agent or it returns no intents above the confidence threshold
KB0996623: [NLU] Training Brazilian Portuguese, Canadian French, Danish, Finnish, Norwegian and Swedish NLU Models do not complete and the NLU Model training request in table [ml_solution] eventually gets timed out, when it should take 15 minutes to complete.
KB1001122: [NLU] How to compare NLU Model utterance testing between DEV and TEST instances and get the same prediction results in NLU Workbench or NLU Batch Testing, when making continuous development changes to the NLU Model on DEV
KB1002365: [NLU] Setup topics should not show up when doing a search in Virtual Agent. The utterance 'Email' triggers the "Personalized Greeting" topic and when a user types in 'Incident', it invokes the "Closing" topic instead of the expected NLU intent/topic
KB1002559: Best Practice: When to use single or multiple NLU Models in your Custom Scoped Applications and how many intents / utterances are supported in a single NLU Model.
KB1047978: [NLU] How to handle an upgrade to a new version of the NLU Service for your NLU Models when upgrading your instance.
KB1120011: NLU Model training throws ERR106: Failed to get referenced solution for solution name [ml_sn_sn_itsm_nlu_global_xxx] version 1. reason - Unable to retrieve dependent fuzzy matcher solution: ml_x_snc_gloval_global_xxx] version

NLU Tools:

We have also released NLU tools to help our customers implement their custom NLU Models, as follows -

In the NLU Workbench, we have Test panel feedback to provide feedback on test utterances that do not return the correct intent.
We also have the NLU Expert Feedback Loop application, which will provide feedback on Virtual Agent chat log utterances to help the system continuously learn and to better predict user input. Once every 30 days, the system pulls up to 300 utterance samples from VA chat logs to the Expert Feedback Loop. The utterances are selected for feedback based on how well they represent all the utterances in the logs. Every utterance sampled from VA chat logs has a suggested intent picked by the system. Once the feedback has been generated, your goal is to review each utterance and mark its relationship status to a given VA intent. You mark each utterance with either the NLU_Match, Mismatch, or Unsure value. After you've marked and saved at least 100 utterances, you can use the marked data to run the "Enhance test set + optimize model" capability to further improve your NLU Model performance based on actual utterances used in Virtual Agent.
Use NLU Model Performance to see how well your models predicted intents in Virtual Agent (VA) based on end-user confirmation. To run a performance analysis, click the Unsupported utterances tab. This section of the UI shows rows of expandable clusters containing VA utterances where NLU didn't make a topic prediction, or where the VA end-user confirmed that the predicted topic was incorrect. The next thing you want to do is to click Expert Feedback Loop. This action takes you to the NLU Expert Feedback Loop application where you review and provide feedback on the utterances that were pulled in from VA.
The Cross-model Conflict Review will identify conflicting intents within or across models so you can take corrective actions, resolve such conflicts, and improve your NLU performance.
Use the Intent Discovery application to help identify opportunities for incident deflection.