252 views

Description

The MOOG Integration with ServiceNow (incident.MOOG) has a single script include, MOOGRestAPIInterface. The interface is used by a SOAP integration that uses a rotating session token for authentication. To store the value of the session token the code uses the incident.moog.session.token ServiceNow system property. The property is frequently updated. Updating properties causes a system-wide cache flush that results in performance loss as the entire cache must be rebuilt.

The following messages will appear in the system logs:
2016-01-18 19:10:03 (057) SOAP-thread-4 SYSTEM Cache synchronize catalog @ 2016-01-18 19:10:02, for catalog: filesByUpdateName, key: sys_properties_986534d4402611009f4bdfd2128e4c46, triggered by system_id: appXXXXX.XXX.service-now.com:XXXX021

Steps to Reproduce

The Moog Integration can cause continual cache flushes leaving the instance susceptible to performance issues. These cache flushes are full cache flushes and triggered by updates to the incident.moog.session.token system property.  The following messages appear in the logs:

2016-08-24 13:08:01 (197) worker.7 worker.7 WARNING *** WARNING *** Starting cache flush
2016-08-24 13:08:01 (197) worker.7 worker.7 Flushing cache
2016-08-24 13:08:03 (620) worker.7 worker.7 Loading properties from DB
2016-08-24 13:08:03 (671) worker.7 worker.7 WARNING *** WARNING *** Cache flush complete
2016-08-24 13:08:03 (672) worker.7 worker.7 Loading properties from DB
2016-08-24 13:08:03 (735) worker.7 worker.7 Slow business rule 'Properties change' on sys_properties:incident.moog.session.token, time was: 0:00:02.53

The cache flush propagates to all nodes.  While one cache flush will not result in instance-wide performance degradation, doing flushes many times in a row can cause performance issues.  The Moog Integration can update the incident.moog.session.token property 300+ times in an hour, so correlating when the performance issues are being seen on the node to when these cache flushes are being triggered is important.

Note – Because when a cache is flushed all queries go to the database until the cache is rebuilt, cache flushes can result in performance issues (high response times, high CPU on application nodes, and high CPU on database servers).

Workaround

Set the ignore_cache field to true for the incident.moog.session.token system property.

Note – sys_properties records should not be used for values that will be frequently changed (that is, more than once a month).

 

 


Related Problem: PRB660693

Seen In

Eureka Patch 11 Hot Fix 4
Helsinki Patch 1

Associated Community Threads

There is no data to report.

Article Information

Last Updated:2018-05-08 07:12:23
Published:2017-01-31