4616 views

Description

Setting the term config table property to glide.db.query.term.table_limit to 0 results in cache thrashing on sys_term_table_cache and very high flush count. When this property is set to zero, the related property, glide.cache.size.sys_term_table_cache, is created with a value of 0 during the upgrade to Kingston. When the sys_term_table_cache value is set to 0, the instance may experience significant performance degradation.

 

 

Steps to Reproduce

  1. Set the system property glide.db.query.term.table_limit to 0

  2. Restart the instance node. A static initializer will then be setting the cache size.

  3. Issue any arbitrary updates, for example against an incident.

  4. Look at /xmlstats.do?include=cache for the entry for sys_term_table_cache.
    Observe that entries are always 0 and flush count climbs with every update.
    Periodic refreshes of /threads.do on a busy instance likely show threads on: com.glide.db.query_rewrite.term.TermTableStorage.get(TermTableStorage.java:54)

 

Workaround

If the instance is affected by this issue, please execute the workaround listed below. This will correct for the issue introduced in PRB1267268, which is documented in KB0683327, and ensure the cache configuration is set correctly. The effect of the workaround will depend on when it is applied.

  1. If applied before upgrading to Kingston, the script will take two actions. Neither of these will have any effect before the instance is on Kingston.
    1. Disable term optimization
    2. Set the Term Optimization cache value to 1, ensuring it is sized correctly after upgrade
  2. If applied during the upgrade to Kingston, the script will prevent the upgrade delay experienced by some customers. If you do not experience an improvement in performance, please contact ServiceNow Customer Support.
  3. If applied after the upgrade to Kingston, the script will apply the appropriate fix, depending on the instance version.

 

The following script is intended to be executed via Scripts - Background (/sys.scripts.do).

  (function() {
    var term_limit_property = 'glide.db.query.term.table_limit';
    var table_cache_property = 'glide.cache.size.sys_term_table_cache';
    var version = gs.getProperty('glide.war');

    var results = {
      'table_limit_before': gs.getProperty(term_limit_property, ''),
      'table_cache_before': gs.getProperty(table_cache_property, ''),
    };

    // Determine Kingston patch.
    var test_patch = function(war) {
      var patchNumber = 0;
      var patchIndex = war.indexOf("patch");
      if (patchIndex > 0) {
        var patch = war.substring(patchIndex, patchIndex + 6); // eg patch5
        patchNumber = patch.substring(5); // eg 5
      }
      if (patchNumber < 5)
        return true;
      else
        return false;
    };

    // Execute the fix for PRB1267268
    if (results.table_limit_before != 0) {
      if (version.indexOf('kingston') > 0) {

        // if before Kingston Patch 5 (or on Kingston but not a patch), then execute stopOptimizing
        if (test_patch(version)) {
          var gr = new GlideRecord('sys_term_config');
          gr.query();
          results.term_config_count = gr.getRowCount();
          while (gr.next()) {
            results.stopOptimizing = true;
            new SNC.GlideTermTableAPI().stopOptimizing(gr);
          }
        }
      }

      // if pre-Kingston, create the property and set it to '0'
      else if (version.indexOf('kingston') == -1) {
        gs.setProperty(term_limit_property, 0);
        results.table_limit_after = 0
      }

      // did not run either audit operation
      else {
        results.error = 'Not able apply fix for PRB1267268';
      }
    }

    // Execute the fix for PRB1282065
    if (results.table_cache_before == 0 || !results.table_cache_before) {
      gs.setProperty(table_cache_property, 1);
      gs.cacheFlush('sys_term_table_cache');
      results.table_cache_after = 1;
    }

    gs.print(new JSON().encode(results));
  })();

 

 


Related Problem: PRB1282065

Seen In

There is no data to report.

Intended Fix Version

Kingston Patch 7
London

Safe Harbor Statement

This "Intended Fix Version" information is meant to outline ServiceNow's general product direction and should not be relied upon in making a purchasing decision. The information provided here is for information purposes only and may not be incorporated into any contract. It is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. The development, release, and timing of any features or functionality described for our products remains at ServiceNow's sole discretion.

Associated Community Threads

There is no data to report.

Article Information

Last Updated:2018-06-18 07:27:04
Published:2018-06-13