183 views

Description

A large number of inserts into the syslog table where message starts with "identification_engine" or source is "identification_engine".

Release or Environment

All currently supported environments.

Cause

The main causes are:

  1. Configuration of identification engine log level
  2. Configuration of identifier
  3. Data source sending "incomplete" data to identification engine

Resolution

The resolution will be a combination of the following:

  1. Configure identification engine log level
  2. Configure identification identifier
  3. Get data source to provide "complete" inputs

Configuration of identification engine log level

Property glide.cmdb.logger.source.identification_engine determines the log level for the IRE. A lot of data will be sent to the syslog table if this property is set to debug, or debugVerbose.

See document:

There is an option to write Identification and Reconciliation Engine (IRE) logs to the node only or to node and syslog. Property glide.cmdb.logger.use_syslog.identification_engine determines if the logs should be written to syslog. Setting the value to false will stop the IRE from inserting into the syslog table.

Classes of interest:

  • com.snc.cmdb.logging.LoggerFactory.java
  • com.snc.cmdb.identify_reconcile.IdentificationEngine.java

Configure identification identifier & data source input

If the bulk of the inserts are for the same class (the syslog message will often contain the class), then the issue is likely due to an identifier which may need more rules configured or with the data source input.

The identifier uses a set of rules to determine the uniqueness of a CMDB CI. These rules use fields in the target table to find the CI. An identifier which has a single rule which uses field "name" would therefore need a payload which contains "name" field and value. An error would be thrown if such payload did not have "name" field.

Example error:

identification_engine : MISSING_MATCHING_ATTRIBUTES In payload missing minimum set of input values for criterion (matching) attributes from identify rule for table [cmdb_ci_docker_image]. Add these input values in payload item '{"className":"cmdb_ci_docker_image","values":{"skip_sync":"false","operational_status":"1","size_bytes":"2413","monitor":"false","can_print":"false","cost_cc":"USD","discovery_source":"ManualEntry","install_status":"1","created_by_qualys":"false","image_digest":"b194f77c4cbf448b01cdf737","name":"openshift-deploy","unverified":"false","fault_count":"0"}}': no thrown error

From the above error, either the identifier needs to be configured to use one of the fields already returned in the payload, or the datasource needs to provide a payload with values used by the IRE. The team managing such data would need to decide which option best fits their setup.

Note: A payload may bring dependent CIs. Some payloads may have thousands of such CIs. If the main CI payload throws an errors, so will each dependent payload. The error would contain "since its depends on payload item". In such cases, focus on fixing the "main" CI payload first.

Problems

PRB1395710If there is an IRE error on the main CI item, IRE logging causing high memory usage on the nodes

Additional Information

Article Information

Last Updated:2020-11-10 12:51:34
Published:2020-06-25