Notifications

70 views

Description

The DB CPU spikes or Read Replica lag occurs during active discovery schedule. Discovery scans on multiple IP Addresses of the same device went through full cycle leading leading to redundant DB operations. In theory, there should be only one IP Address scan went through the full Discovery cycle, the rest of the IP Addresses for the same device in a discovery schedule should be detected as extra/duplicated IP Addresses and halted. This behavior is observed during Probe based discovery. However, during Pattern based discovery, this did not happen.

Cause

Out of the box, Pattern based discovery do have features to detect extra/duplicated IP Addresses. The behavior is controlled by a couple of properties:

glide.discovery.ip_based.active = false
glide.discovery.device.duplicate.ip.optimization = true

 

However, an upgrade script checks to see if "Discovery - IP Based" plugin is enabled, if it is set the "glide.discovery.ip_based.active" to true. Probe-to-Pattern migrated instances will always have this property set to true. "glide.discovery.device.duplicate.ip.optimization" property is always true out of the box.

 

Because the property "glide.discovery.ip_based.active" is true for probe-to-pattern migrated instances, Discovery does not attempt extra/duplicate IP Address detection. This allows scans on all IP Address of the same device in the same discovery schedule to go through full cycle and needlessly runs redundant operations on DB leading to much higher DB load.

 

*** NOTE: The extra/duplicate IP Address detection feature make use of the "install_status" field on CI tables. This field is defaulted to 1 OOB. If the default value is removed or modified, this feature will not work.

Resolution

Set the following System Property:

glide.discovery.ip_based.active = false
glide.discovery.device.duplicate.ip.optimization = true

 

Make sure default value exists for at least IP Address or Network Adapter records.

Article Information

Last Updated:2019-08-20 13:25:14
Published:2019-08-20