29 views

Symptoms


The customer was executing a text search against the kb_knowledge table but was getting the search results returned in an unexpected order.
 
 
The customer had configured a number of fields of the kb_knowledge table to have different weightings using the ts_weight field attribute:
 
https://docs.servicenow.com/bundle/kingston-platform-administration/page/administer/search-administration/task/t_ControlMatchRelevanceByField.html
 
 
With the specified field weightings, the results were not being returned in the order expected eg. most relevant article first in the results list.
 
 
Debug of the text search revealed the scoring of each of the records returned by the search.
 
for example:
 
other (not gif, sql)14:15:20.603: <<<<< START Display relevency scores for table: kb_knowledge >>>>>
other (not gif, sql)14:15:20.606: 1: Score for document 1631afebdbf91b00afe666a25b9619bd is: 255.0
other (not gif, sql)14:15:20.606: 2: Score for document 876fdbe7dbf91b00afe666a25b96193a is: 255.0
other (not gif, sql)14:15:20.606: 3: Score for document e14fdf2bdbf91b00afe666a25b9619bd is: 255.0
other (not gif, sql)14:15:20.606: 4: Score for document 1f8e93e7dbf91b00afe666a25b961919 is: 255.0
other (not gif, sql)14:15:20.606: 5: Score for document ec7943a3dbb91b00afe666a25b9619e5 is: 255.0
other (not gif, sql)14:15:20.606: 6: Score for document 4c5e17a7dbf91b00afe666a25b96199d is: 252.0
other (not gif, sql)14:15:20.606: 7: Score for document af7ddb27dbf91b00afe666a25b961969 is: 250.0
other (not gif, sql)14:15:20.606: <<<<< END Display relevency scores for table: kb_knowledge >>>>>
 
 
The scores were being capped at a maximum value of 255 and thus the most relevant record were not being displayed at the top of the list. The results were effectively random as they all had the same score value.
 
It was expected that the scoring of the records would have followed the process documented here:
 
https://docs.servicenow.com/bundle/kingston-platform-administration/page/administer/search-administration/concept/c_DocumentScoring.html
 
In the customer's case they did not have inverse document frequency (IDF) enabled.
 
 

Release


Jakarta

 

 

Cause


After engaging with development it was identified that customer was encountering the following problem:

 

PRB1237035 Knowledge results always appear above other categories because its ir_query_score is consistently higher

 

This is fixed in London Release.

 

Resolution


The workaround was to set the following attribute on the kb_knowledge table's collection entry in the dictionary:

text_index_attachment_displayed = true

 

This attribute is currently undocumented.

 

Development advised "That attribute (text_index_attachment_displayed)  specifies if attachments will be displayed in text search results."

 

The folloiwng documentation enhancement request has been raised:


DOC61116 collection attribute text_index_attachment_displayed

 

Note: It was not necessary to re-regenerate the text index for the kb_knowledge table after making the above change.

 

After adding the attribute the customer was able to see the most relevant articles at the top of the search results.

 

Debug of the text search revealed that the scoring was no longer being capped at the value 255:

other (not gif, sql)12:03:09.142: <<<<< START Display relevency scores for table: kb_knowledge >>>>>
other (not gif, sql)12:03:09.145: 1: Score for document e14fdf2bdbf91b00afe666a25b9619bd is: 510.0
other (not gif, sql)12:03:09.145: 2: Score for document ec7943a3dbb91b00afe666a25b9619e5 is: 505.0
other (not gif, sql)12:03:09.145: 3: Score for document 1f8e93e7dbf91b00afe666a25b961919 is: 500.0
other (not gif, sql)12:03:09.145: 4: Score for document 1631afebdbf91b00afe666a25b9619bd is: 255.0
other (not gif, sql)12:03:09.145: 5: Score for document 876fdbe7dbf91b00afe666a25b96193a is: 255.0
other (not gif, sql)12:03:09.145: 6: Score for document 4c5e17a7dbf91b00afe666a25b96199d is: 252.0
other (not gif, sql)12:03:09.145: 7: Score for document af7ddb27dbf91b00afe666a25b961969 is: 250.0
other (not gif, sql)12:03:09.145: <<<<< END Display relevency scores for table: kb_knowledge >>>>>

 

Article Information

Last Updated:2018-08-17 07:44:22
Published:2018-08-03