Description
It's been seen that some customer email systems can send us multiple emails with the same Message-ID. By default the instance will save the first email to sys_email and then silently drop any subsequent ones with the same message ID.
Release or Environment
All current ServiceNow versions
Instructions
To see the emails with duplicate Message ID's that are being dropped ensure that System Property glide.email.debug is set to type true|false and value true.
Emails that are dropped due to being duplicates will be logged as, for example:
2020-01-22 10:07:06 (056) worker.5 worker.5 txid=12f345bc1ba6 Message UID='00123f4e56789745' Message-ID='<AJGHEYTISMBTHAYAAAAAAAAAMEul/ioeqdYUIO1jyq7tY3CgAAAEAAAAICPx8Ch0U1Bu2IX+wLq2D0BBBBB==@example.com>' was previously read and will be ignored.
Workarounds
This behaviour can be changed by setting System Property glide.email.allow_duplicate_message_ids to type true|false and value true. But this has implications, in particular that matching of reply emails based on the In-Reply-To header will pick up a random email from the duplicates.
After this workaround has been applied the emails with duplicate Message ID's will be stored on Email[sys_email] as normal. However note that when searching on the Email[sys_email] table for emails with duplicate Message ID you *must* search by Type (e.g Received) and Message ID, otherwise you won't use the OOB index on type,message-id.
Additional Information
The ServiceNow product by default assumes that Message ID's are unique because that's what the relevant specification says:
3.6.4. Identification Fields
Though listed as optional in the table in section 3.6, every message
SHOULD have a "Message-ID:" field. Furthermore, reply messages
SHOULD have "In-Reply-To:" and "References:" fields as appropriate
and as described below.
The "Message-ID:" field contains a single unique message identifier.