How to troubleshoot duplicate CMDB CI records - Support and Troubleshooting

How to troubleshoot duplicate CMDB CI records Issue .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } span { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } h2 { font-size: 24pt; font-family: Lato; color: var(--now-color--text-primary, black); } h3 { font-size: 18pt; font-family: Lato; color: var(--now-color--text-primary, black); } h4 { font-size: 14pt; font-family: Lato; color: var(--now-color--text-primary, black); } a { font-size: 12pt; font-family: Lato; color: var(--now-color--link-primary, #00718F); } a:hover { font-size: 12pt; color: var(--now-color--link-primary, #024F69); } a:target { font-size: 12pt; color: var(--now-color--link-primary, #032D42); } a:visited { font-size: 12pt; color: var(--now-color--link-primary, #00718f); } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } Troubleshoot duplicate configuration item (CI) records in the CMDB created by the Identification and Reconciliation Engine (IRE). This article explains how the IRE identifies duplicates, common causes of duplicate CI creation, and how to resolve them.

The IRE provides a centralized framework for identifying and reconciling data from different data sources. It helps maintain the integrity of the CMDB when multiple data sources create and update CI records. The IRE can be tailored to meet the needs of your organization through identification rules, reconciliation rules, and relationships. For more information, see CMDB Identification and Reconciliation.

Depending on the configuration, duplicate records can be created. A duplicate record, as defined by the IRE, is a record created when a CI should have been updated instead based on the identification rules. The IRE uses identification rules to find CIs according to the CMDB hierarchy. To see what rules apply to a specific class, go to Configuration > CI Class Manager .

For example, the following image shows the default identifier used for Windows servers. This identifier has the following characteristics:

Derived — Inherited from a parent class ( cmdb_ci_hardware ). Independent — Does not depend on relationships to other CIs. Correlation ID — First matching criteria. Given this identifier, two Windows servers created by the identification engine with the same Correlation ID would be considered duplicates.

Note : The CMDB supports domain separation. Servers created in their respective domains with the same Correlation ID are considered unique. For more information, see Domain separation in Configuration Management Database (CMDB).

The following image shows a custom identifier created for Windows Server where:

Derived is no longer present. Only serial number is used to determine uniqueness. With this configuration, two Windows servers with the same Correlation ID would not be considered duplicates.

Note : Creating new identifiers may also require related entries to be created. If a custom identifier is necessary for a child class of cmdb_ci_hardware , see How to copy the Hardware Rule CMDB Identifier for specific Sub-Classes.

The IRE uses the following logic for independent CIs:

A payload is passed to the IRE. Validate the payload CIs and relationships. Valid : Continue. Not valid : Log errors and stop processing. Loop through the identification rules and query the CMDB. CI found: Is the CI active given the inclusion rules for this class? Yes : Are there reconciliation rules for this class? Yes : Do the reconciliation rules allow this source to update the CI? Yes : Update CI fields. No : Do not update CI. No : Update CI fields. No : Create CI. No CI found : Are there more identification rules to attempt? Yes : Continue to the next identification rule. No : Create CI. Note : As an exception, vCenter discovery does not use the IRE. For more information, see vCenter Discovery.

For more details on how the IRE works, see How the CMDB Identification and Reconciliation Engine processes a CI payload.

Dependent CIs Dependent CIs also use the identification rules. However, dependent relationship rules define the dependency structure of CI types and the relationship types in service definitions, helping in CI identification and construction of business service maps. During CI identification, the pair of CIs being examined must satisfy at least one hosting rule.

For example, the following image shows the hosting rule for Kubernetes Pod from Configuration > Identification/Reconciliation > Metadata Editor :

The following image shows the identification rule for Kubernetes Pod from Configuration > CI Class Manager :

Based on the combination of the identification rule and hosting rules, a Kubernetes pod must meet the following conditions to be unique:

Have a unique Kubernetes UID. Be the only Kubernetes pod with a "Contained by" relationship to the Kubernetes cluster. The following two Kubernetes pods would be unique:

Kubernetes Pod A — UID Y with a "Contained by" relationship to Kubernetes cluster A. Kubernetes Pod B — UID Y with a "Contained by" relationship to Kubernetes cluster B. Both pods have the same UID but are contained by different Kubernetes clusters.

For more information, see the following resources:

CMDB dependent relationship rules Create dependent relationship rules Release .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } span { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } h2 { font-size: 24pt; font-family: Lato; color: var(--now-color--text-primary, black); } h3 { font-size: 18pt; font-family: Lato; color: var(--now-color--text-primary, black); } h4 { font-size: 14pt; font-family: Lato; color: var(--now-color--text-primary, black); } a { font-size: 12pt; font-family: Lato; color: var(--now-color--link-primary, #00718F); } a:hover { font-size: 12pt; color: var(--now-color--link-primary, #024F69); } a:target { font-size: 12pt; color: var(--now-color--link-primary, #032D42); } a:visited { font-size: 12pt; color: var(--now-color--link-primary, #00718f); } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } All releases

Cause .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } span { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } h2 { font-size: 24pt; font-family: Lato; color: var(--now-color--text-primary, black); } h3 { font-size: 18pt; font-family: Lato; color: var(--now-color--text-primary, black); } h4 { font-size: 14pt; font-family: Lato; color: var(--now-color--text-primary, black); } a { font-size: 12pt; font-family: Lato; color: var(--now-color--link-primary, #00718F); } a:hover { font-size: 12pt; color: var(--now-color--link-primary, #024F69); } a:target { font-size: 12pt; color: var(--now-color--link-primary, #032D42); } a:visited { font-size: 12pt; color: var(--now-color--link-primary, #00718f); } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } CIs in different domains are not considered duplicates even if their data is the same.

Duplicates may be created when any of the following factors prevent the identification engine from finding an existing CI:

Inclusion rules Query business rules Access control lists (ACLs) The IRE queries the CMDB in the context of the user calling the IRE. If any of these factors prevent the IRE from finding a CI, a new CI is created instead of updating the existing one. A common symptom is a duplicate CI created every time the same server is discovered or a call is made to the IRE API.

Additionally, depending on the release, the IRE does not identify CIs when:

The duplicate_of field is not empty. The discovery_source field is set to Duplicate . Slow business rules can contribute to duplicate dependent CIs when multiple payloads with the same dependent CIs are processed simultaneously. This can occur when discovering the same CI in separate discoveries at the same time, or when payloads in the same discovery contain the same dependent CI in more than one payload. This is a race condition and is not common.

Resolution .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } span { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } h2 { font-size: 24pt; font-family: Lato; color: var(--now-color--text-primary, black); } h3 { font-size: 18pt; font-family: Lato; color: var(--now-color--text-primary, black); } h4 { font-size: 14pt; font-family: Lato; color: var(--now-color--text-primary, black); } a { font-size: 12pt; font-family: Lato; color: var(--now-color--link-primary, #00718F); } a:hover { font-size: 12pt; color: var(--now-color--link-primary, #024F69); } a:target { font-size: 12pt; color: var(--now-color--link-primary, #032D42); } a:visited { font-size: 12pt; color: var(--now-color--link-primary, #00718f); } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } Inclusion rules Go to Configuration > Identification/Reconciliation > Identification Inclusion Rules . Find the inclusion rule that applies to the class or one of its parent tables. Adjust the inclusion rule accordingly. Warning : Creating an inclusion rule for a parent class such as cmdb_ci_hardware with conditions that include "Class" can cause duplicate CIs. The inclusion rule applies to all child classes, so any child class passed to the IRE that is not included in the condition is not found by the engine. Avoid using "Class" as a condition in an inclusion rule.

For more information, see Create health inclusion rule.

Query business rules and ACLs Find custom query business rules and ACLs for the affected table and disable them. Try to reproduce the issue. If disabling a business rule or ACL resolves the issue, that confirms the root cause. Review the custom configurations and adjust accordingly. Note : By default, no query business rules or ACLs in the CMDB tables affect the IRE.

For more information, see the following resources:

Debugging business rules ACL debugging tools Slow business rules - dependent CIs Improve the performance of the business rule. If possible, run the business rule asynchronously. In the pattern, add a step to remove duplicates before they are sent back to the instance. Discovery only : Set the glide.discovery.multi_page_serial_mode property to true so that multipage payloads are processed in serial. This prevents the creation of duplicate CIs by Discovery. General troubleshooting tips Have clear steps to reproduce the issue. Use the smallest payload set to simplify troubleshooting (include only the necessary CIs). Test the payload with different users if possible to verify different permissions. In CI Class Manager, check the identification rules to confirm the CI should be considered a duplicate. In CI Class Manager, check for inclusion rules. Debug IRE execution using a background script to check whether a business rule could be affecting CI queries. For more information, see How to debug the Identification and Reconciliation Engine using background scripts. Related Links .ns-kb-css-body-editor-container { p { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } span { font-size: 12pt; font-family: Lato; color: var(--now-color--text-primary, #000000); } h2 { font-size: 24pt; font-family: Lato; color: var(--now-color--text-primary, black); } h3 { font-size: 18pt; font-family: Lato; color: var(--now-color--text-primary, black); } h4 { font-size: 14pt; font-family: Lato; color: var(--now-color--text-primary, black); } a { font-size: 12pt; font-family: Lato; color: var(--now-color--link-primary, #00718F); } a:hover { font-size: 12pt; color: var(--now-color--link-primary, #024F69); } a:target { font-size: 12pt; color: var(--now-color--link-primary, #032D42); } a:visited { font-size: 12pt; color: var(--now-color--link-primary, #00718f); } ul { font-size: 12pt; font-family: Lato; } li { font-size: 12pt; font-family: Lato; } img { display: ; max-width: ; width: ; height: ; } } Using identification simulation CMDB APIs (CMDB SDK)

IdentificationEngineScriptableApi - Global