If a MID Server has one or more extension listeners, and the MID loses connectivity to the instance and the MID is part of a cluster, the failover business rule will restart the listener on another MID in that cluster. If, at a later time, the MID that originally lost connectivity comes back online, it will still be running the extension listener, resulting in duplicate events being sent to the instance.
Sometimes when failover occurs in a MID cluster for SNMP traps, you might see duplicate events in em_event. When checking the MID server threads, you see that "Trap" threads are running on both MID servers instead of just the active one defined in the SNMP Trap Extension. Other extensions could also be affected in similar ways including vCenter, Event Management, and MID Web Server.
Steps to Reproduce
- Cause failover of "Failover" MID Server cluster for SNMP Trap Extension (ideally by MID connectivity interruption to instance i.e., temporarily restrict firewall rules).
- Verify the failover. Extension started on other cluster member MID Server.
- Resume connectivity of previously primary MID Server.
- Validate that trap threads are running on both MID Servers.
- Send SNMP traps to both MID Servers and note parallel processing. Duplicate events are seen in em_event.
Restart the MID that is still listening.
Related Problem: PRB1326217