Upgrade FAQ (Frequent queries recieved as tickets to tech support)
Objective of this KB is to answer a few frequent requests/questions Tech support receives in related to the upgrades. Please append the generic questions in the comments so that we can keep this KB updated. This information is generic and if you have follow-up questions, please create a ticket to us.
The customizations on your instance can affect the below behavior and this is from an OOB perspective.
Upgrade didn’t start and it went past the planned start time on the HI change?
Planned start of the change determine when the Assigned WAR version of the instance should be changed in HI instance record.
Upgrade is triggered by the Upgrade Job on the instance.
OOB; this job is set to run in an hourly interval.
The upgrade can take up to 1 hour to trigger on the instance from the planned start time of the HI change.
Only when the upgrade is triggered on the instance, Actual Work start time is updated on the HI Change.
Check this information by going to the below URL.
Please make sure the value on System ID is blank. If there is a value, please make sure that this is selected to –None--
Hardcoding the System ID is a bad approach and there are chances that the node selected can be a far node (from secondary DC) and the upgrade can run for long.
Please refer to this document for Upgrade monitor to monitor the upgrade progress.
How long will my upgrade run?
This is a broader question and are unable to porvide an ETA for this. Every instance is different with Organizational specific customizations and the upgrade time can vary based on the amount of data/customizations on the instance.
Best process to dry run my upgrade?
To get an ETA/behavior of a PROD upgrade is very important.
To record the correct timelines/behavior there are 2 options:
- Option 1: Initiate a clone from PROD to SUBPROD including (Default is EXCLUDE) attachments, audit and log and then upgrade the SUB PROD instance. Including the above specified tables are very important as these are the largest tables on an instance and index creation/updates to this table can take a major chunk of the total upgrade time. If they are excluded, that time cannot be factored into the planned upgrade activity.
- Option 2: Request customer support for a restore of your PROD instance to the SUB PROD instance so that the SUB PROD instance can be an exact replica. SUB PROD can be upgrades and this should give you a definite ETA on the upgrade run time.
Upgrades should be DRY RUN multiple times. If we do a proper testing of upgrade on SUB PROD’s the upgrade on PROD will go fine and the support is available 24X7 to look into any unforeseen issue.
Can we take an ad-hoc backup prior to the upgrade?
Unfortunately, there is not an option to run an ad-hoc backup.
We take separate backups for Primary and Secondary DB (where available).
The backup cycle consists of four weekly full backups and the past 6 days of daily differential backups that provide 28 days of backups. All backups are written to disk, no tapes are used and no backups are sent off site. All the controls that apply to live customer data also apply to backups. If data is encrypted in the live database then it will also be encrypted in the backups."
Your Data is safe with us. In a most unlikely scenario of a data corruption in PROD, we have an option to do a POINT IN TIME restore of PROD instance to your SUB PROD instance by which we can bring back the data on PROD instance to that specific time for data comparison (to a specific minute). Point in time restore is done by restoring the instance to the last available backup and recreating the remaining data specified from the BIN logs/Transaction Logs. This process can take more time. But we can bring the instance to any point/date-time we need in previous 2-3 days (BINLOG retention is 3 days).
PIT restore is a manual process involving multiple teams and this will be initiated in Critical conditions and cannot be treated as a Failover/Recovery/Back out Method. We do not do PIT restore directly on PROD and we will only fix forward and the PIT will be used only for Data comparison and Data recovery.
What happens to my instance during an upgrade?
During an upgrade, users can still be logged on to the instance and work as usual. There will be minimal impact and user connection will be reset once when the Node to which user is connected to is getting upgraded. Record creation and records created will not be Impacted by the upgrade. Only the Scripts and the Schema will be modified via upgrade.
One node will be running the whole process by Upgrading the node then triggering the upgrade of the Schema/DB. In parallel, it will run the upgrade on other nodes and after the upgrade, nodes will be restarted resetting the connections.
So once the nodes are restarted, the nodes will be in Newer Version and the DB will not be in Newer version till the Database upgrade is completed. Even though record creations are not interrupted; my personal recommendation will be to restrict users during the upgrade. By this the system can use all the available resources for the upgrade activity.
Features impacted during an upgrade
How to restrict users during an upgrade
There is no OOB option available to restrict users during an upgrade. There are a few workarounds and customization to achieve this and this should be implemented with your Developers/Partners and out of Tech support scope.
Few options available are as follows:
- If your instance has Single sign on implemented, you can restrict the access via the portal/identity provider for SSO/SAML.
- You can also run a custom script to logout all logged in users prior to the change/upgrade window.
- You can also create a Local Admin account and only admin users can access the Instance with a Local Account of the instance skipping the SAML/SSO login
There is a planning and checklist for upgrade relevant to Kingston version can be found here:-
Overview and Phase wise documents can be found here:-
Schedule/Modify/Cancel the upgrade
Monitoring the upgrade from the backend local host log is a non-realistic approach as there will be up-to 150 lines created in the backend per second.
The best way to Monitor the upgrade is via the Upgrade monitor window. We Tech support also use this window to monitor the upgrades because of the above said reason.
Upgrade monitor will tell the number of plugins remaining and an estimate of what is upgrading and an estimate of remaining records per plugin as and when it run.