Notifications

992 views

Description

When service mapping discovery is executing multiple/complex business services, for example All Services, the multiple internal service re-computation jobs might slow down the discovery job processing. This can cause a bottleneck to the schedule jobs and caused scheduler overload, discovery task timeouts, and affect other functionalities on the ServiceNow instance. 

Symptoms

  1. The stats.do page of the instance, is showing multiple scheduler workers, having long running schedule with ASYNC: Script Job
  2. And the threads of these scheduler workers contains:
    com.snc.sw.mapping.ServiceMappingFactory.jsStaticFunction_recomputeLayer(ServiceMappingFactory.java:269)

Both conditions have to match, in order to qualify for this issue.

 

Steps to Reproduce

 

Complete a service mapping discovery for multiple services, or all services.

Workaround

Immediate Relief

  1. Import the Business Rule from file sys_script_f73da5fc0fd972402df991dbe1050e33.xml attached to this PRB.

    This business rule sets all incoming re-computation job priority to 150 in sys_trigger table, instead of default 100, and hence it is lower than discovery, using 105 – 110 priority. This change gives the discovery jobs higher precedence and processed ahead of the re-computation jobs.

  2. Use the following script to kill any queued re-computation jobs, if needed

// set to true to actually clean up the jobs var update = false; var gr = new GlideRecord("sys_trigger"); gr.addQuery("script", "SNC.ServiceMappingFactory.recomputeLayer(current);"); gr.query();
while (gr.next()) {
gs.log(gr.getUniqueValue());
if (update) {
gr.setValue("script", "gs.log('skip');");
gr.update();
}
}

Middle Term Relief

Import the update set model_workers_limit_update_set.xml attached to this PRB.

The update set contains the following contents:

  1. Business Rule (Update Service Mapping Job Priority)
  2. Business Rule (Update Model Reschedule)
  3. Script include (ModelRecompTrigger)

The limit is currently hardcoded to 4 workers, and the exceeding tasks are postponed by 1 minute.
To change those values, if necessary, modify the scripts above.

Long Term Solution

Please upgrade to version higher than Helsinki Patch 10, for Helsinki family version, Istanbul Patch 5, for Istanbul family version, or Jakarta version.


Related Problem: PRB832751

Seen In

There is no data to report.

Fixed In

Helsinki Patch 10
Helsinki Patch 11
Istanbul Patch 5
Istanbul Patch 7
Jakarta
Jakarta Patch 1
Kingston

Associated Community Threads

There is no data to report.

Article Information

Last Updated:2018-12-03 11:32:04
Published:2017-08-25
model_workers_limit_update_set.xml