50 views

Configuring Stop Words to Improve Global Text Search




Overview


Stop words can improve system performance and generate more meaningful search results.

When a user initiates a keyword search, the system gathers and scores documents containing the search term. If the term appears in a lot of documents, the search engine must process many results, which takes time. Some words—for example, the, with, but, and similar terms—are so common that in a search query, they would return mostly irrelevant results. So these common terms are designated as stop words and eliminated from searches.

The ServiceNow global text search is already configured with common stop words. However there may be words that are not common in general, yet occur very frequently in a particular ServiceNow instance. In this case, users can customize stop words on an instance at the global and table levels, and set up the system to automatically identify stop words based on their frequency of occurrence, resulting in more effective global text searches.

 

Note: Only Administrators and users with the ts_admin role can configure stop words.



 

 

 

Configuring Global Stop Words


Please see the documentation pages below for the latest guidelines:

Configure a global stop word

 

Global stop words apply to all indexed tables in the instance.

To add a new global stop word or modify an existing word:

    1. Navigate to System Definition > Text Index Stop Words. The current global stop words are displayed.

 

    1. Do one of the following:

        • Search for a new stop word to be added (to verify that it is not already in the list), and if it is not listed, click New.

      • Search for and click an existing stop word to be modified.

 

    1. In the Word field, do one of the following:

        • Enter the new word to be added.

      • Verify or change the existing word.

 

    1. Verify that the Active check box is selected. This indicates the record is active.

 

    1. (Optional) In the Comment field, enter a comment to explain when or why the word was added or modified.

 

  1. Click Submit (to add a new word) or Update (to update an existing word).



To delete a global stop word:

    1. Navigate to System Definition > Text Index Stop Words.

 

    1. Search for and click the stop word to be deleted.

 

    1. Click Delete.

 

    1. Click OK to confirm the deletion.

 

  1. If a message at the top of the list says that all text indexes should be regenerated to apply the stop word change, contact Technical Support to regenerate all indexes.

 

 

Configuring Table-Specific Stop Words


Please see the documentation pages below for the latest guidelines:

Configure a table-specific stop word

 

Stop words can be configured for a particular table. For example, if the term com appears very frequently in the Task table, designating it as a stop word for that table may improve search performance.

To add a new stop word or modify an existing word for a specific table:

    1. Navigate to System Definition > Text Indexes.

 

    1. Click the name of the table to open its text index entry. The table-specific stop words are listed at the bottom of the content frame. Since tables inherit the global stop words, this list will be empty for most tables by default.

 

    1. Do one of the following:

        • Click New to add a new stop word.

      • Click an existing stop word to be modified.

 

    1. Verify that the Index field contains the index name.

 

    1. In the Word field, do one of the following:

        • Enter the new word to be added.

      • Verify or change the existing word.

 

    1. In the Stop mode list, select the stop mode. The recommended stop mode is Neither Index nor Query, which tells the system not to index the word or include it in queries.

 

    1. (Optional) In the Comment field, enter a comment to explain when or why the word was added or modified.

 

    1. Click Submit (to add a new word) or Update (to update an existing word).

 

    1. Make other stop word changes (add, modify, or delete) for this table, as needed.

 

    1. Click Regenerate Text Index.

 

  1. Click OK.



To delete a stop word from a specific table:

    1. Navigate to System Definition > Text Indexes.

 

    1. Click the name of the table to open its text index entry. The table-specific stop words are listed at the bottom of the content frame.

 

    1. Click the stop word to be deleted.

 

    1. Click Delete.

 

    1. Click OK to confirm the deletion.

 

    1. Make other stop word changes (add, modify, or delete) for this table, as needed.

 

    1. Click Regenerate Text Index.

 

  1. Click OK.

 

 

Configuring Automatic Stop Words for a Table


Please see the documentation pages below for the latest guidelines:
 
Enable automatic stop words for a table
 

Automatic stop words work at the table level. Once the frequency of any word in the table reaches a certain threshold—for example, appearing 50,000 times or more—the system automatically designates it as a stop word for that table. By default, the automatic stop words feature is enabled for each table at a threshold of 250,000, but you can adjust the threshold, as appropriate for your instance.

To configure automatic stop words for a table:

    1. Navigate to System Definition > Text Indexes.

 

    1. Click the name of the table to open its text index entry.

 

    1. Select the Auto stop check box. (Clearing this check box disables automatic stop words.)

 

    1. In the Auto threshold field, enter the number of occurrences at which the system will automatically designate any word as a stop word for the table.

 

  1. Click Update.

    When a word’s frequency of occurrence in the table meets or exceeds the Auto threshold level, that word is automatically added as a stop word for the table with the Stop mode of Index but do not Query and a Comment to indicate that the stop word was generated automatically.

Automatic stop words are compiled on a nightly basis by the TS Index Stats scheduled job.

 

Article Information

Last Updated:2018-02-26 02:38:25
Published:2018-02-26