How to determine if you're at risk?

This KB is documenting the approved workaround for PRB1028261 for those potentially at-risk for long running upgrades caused by column adds to em_alert_history table.  Criteria for being at risk:
  • Planning to upgrade from Helsinki to Istanbul+
  • em_alert_history table is greater than 1M records
  • The following columns do not exist and are added on upgrade: correlation_group, correlation_rule_group, group, group_source, impact_services, is_group_alert

Workaround Steps: 

  1. Clean up un-wanted em_alert_history records to reduce the size of the table:
    • Add table cleaner on em_alert_history by importing the sys_auto_flush record attached: sys_auto_flush_62c63f08930002000fb3b9ab357ffb4a.xml
      • Navigate to /sys_auto_flush_list.do, and right click on top of list and Import XML, then import sys_auto_flush_62c63f08930002000fb3b9ab357ffb4a.xml
  2. Trigger the Table Cleaner scheduled job so the scheduled cleanup of em_alert_history is triggered. See the set-up in Step 1
    • Navigate to /sys_trigger_list.do?sysparm_query=GOTOname%3DTable%20Cleaner, open the Table Cleaner scheduled job, and then right-click on the record and Execute Now
      • Alternatively, this job runs once an hour, may allow the scheduler to pick it up and run without intervention. If this is the case, move to the next step to confirm that the job finished running.
    • The clean up of em_alert_history table may take approx. 2 hours. Confirm that Table Cleaner job finished processing:
      • Navigate to /syslog_transaction_list.do?sysparm_query=sys_created_onONToday%40javascript%3Ags.daysAgoStart(0)%40javascript%3Ags.daysAgoEnd(0)%5Eurl%3DJOB%3A%20Table%20Cleaner and then verify that there is a recent record for JOB: Table Cleaner showing that the Table Cleaner job run above completed.
  3. Apply the attached Update Set sys_remote_update_set_f6d8cd66db5bb200b9d5b701ef9619e3.xml adds 6 new columns to em_alert_history. This takes time for the Update Set to be committed as the em_alert_history table is altered to add the new columns.  This runs in the background without impact on the instance functionality.
    • In testing, adding these columns to an em_alert_history table with 10M records took an hour.
  4. After new columns are added and update set completes, go forward with the updated to Istanbul or Jakarta.
Additional Notes:
  • This should be done in Helsinki instance- the problem exists on upgrade from Helsinki to Istanbul/Jakarta
  • This should be tested first on a sub-prod instance before doing so in production


Seen In

There is no data to report.

Associated Community Threads

There is no data to report.

Article Information

Last Updated:2018-03-02 06:50:46