When an instance is upgraded, the MID Server needs to upgrade itself to match and needs to Download Upgrade Files from https://install.service-now.com/ and if it does not have access, it fails to upgrade. The automatic Test MID Server connectivity feature will check for and notify you if the MID Server can't.

 Warning: A MID Server on the wrong version can cause code and data mismatches between MID Server and instance, potentially causing the MID Server to fail to process commands sent it by the instance, or the instance to not process data coming back from the MID Server. APIs related to Validation and encryption Keychains may also not match.

Symptom

AutoUpgrade.3600 Performing pre-upgrade validation tests.
AutoUpgrade.3600 Downloading from https://install.service-now.com/glide/distribution/builds/package/mid-upgrade/2019/07/16/mid-upgrade.mid.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636.preUpgradeCheck.zip
AutoUpgrade.3600 WARNING *** WARNING *** java.net.SocketTimeoutException: connect timed out when posting to https://install.service-now.com/glide/distribution/builds/package/mid-upgrade/2019/07/16/mid-upgrade.mid.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636.preUpgradeCheck.zip
AutoUpgrade.3600 SEVERE *** ERROR *** java.net.SocketTimeoutException: connect timed out when posting to https://install.service-now.com/glide/distribution/builds/package/mid-upgrade/2019/07/16/mid-upgrade.mid.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636.preUpgradeCheck.zip
AutoUpgrade.3600 Downloading from http://install.service-now.com/glide/distribution/builds/package/mid-upgrade/2019/07/16/mid-upgrade.mid.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636.preUpgradeCheck.zip
AutoUpgrade.3600 WARNING *** WARNING *** org.apache.commons.httpclient.ConnectTimeoutException: The host did not accept the connection within timeout of 10000 ms when posting to http://install.service-now.com/glide/distribution/builds/package/mid-upgrade/2019/07/16/mid-upgrade.mid.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636.preUpgradeCheck.zip
AutoUpgrade.3600 SEVERE *** ERROR *** org.apache.commons.httpclient.ConnectTimeoutException: The host did not accept the connection within timeout of 10000 ms when posting to http://install.service-now.com/glide/distribution/builds/package/mid-upgrade/2019/07/16/mid-upgrade.mid.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636.preUpgradeCheck.zip
AutoUpgrade.3600 SEVERE *** ERROR *** Aborting MID Server upgrade due to pre-upgrade check failure: Unable to download updates from install server
AutoUpgrade.3600 Setting mid status to Upgrade Failed
AutoUpgrade.3600 Instance.updateAgentRecordState(), OperationalState=UPGRADE_FAILED

The MID Server form and Issues table will also repeat the error:

Aborting MID Server upgrade due to pre-upgrade check failure: Unable to download updates from install server

All versions since Geneva, with Windows or Linux MID Servers.

For new London or Madrid releases, this would not have been an issue due to defaulting to using the instance as a kind of proxy (mid.download.through.instance=true), meaning direct access to the install server was not necessary. Since London Patch 8, Madrid Patch 3 and New York, the MID Server once again has to have access to the install server (PRB1332088/PRB627019). It is recommended that the property is set to false, and access is provided to the install server, in order to avoid API_INT semaphore exhaustion, MID Servers failing to connect to the instance and shutting down, and extended upgrade times during instance upgrades.

In the following situations, your MID Server computer has no access to the install server and cannot auto-upgrade itself. You should try to resolve these configuration issues internally first, as these are our documented connectivity requirements:

• The instance is on-premise and installed inside the customer network (with no Internet access), and the MID Server also has no internet access at all.
• The instance is hosted in our datacenter, but although the MID Server does have access to the instance, you have not yet arranged for it to have access to our upgrade server: https://install.service-now.com/

This manual procedure fools the MID Server into thinking it has downloaded the files itself already, allowing it to upgrade itself in the normal way, and if necessary must be done for every MID Server after every upgrade or patch of every instance, without fail. 

1) Find the filenames from the Agent log:

On the MID Server computer, check the latest 'AutoUpgrade' or 'StartupSequencer' thread entries in the Agent Log for the "Missing:" ZIP file names:
<install folder>\agent\logs\agent0.log.0

AutoUpgrade.3600 Current packages:
AutoUpgrade.3600 Installed: [mid-core.kingston-10-17-2017__patch0-11-06-2017_11-11-2017_1422.universal.universal.zip, mid-jre.kingston-10-17-2017__patch0-11-06-2017_11-11-2017_1422.windows.x86-64.zip]
AutoUpgrade.3600 Assigned: [mid-upgrade.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636.universal.universal.zip, mid-core.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636.universal.universal.zip, mid-jre.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636 .windows.x86-64.zip]
AutoUpgrade.3600 Missing: [mid-upgrade.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636.universal.universal.zip, mid-core.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636.universal.universal.zip, mid-jre.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636 .windows.x86-64.zip]
AutoUpgrade.3600 Downloaded: []

In that example, the MID Server is still Kingston Patch 0, even though the instance is already upgraded to New York Patch 0 Hotfix 2, and the missing files in this example are:

mid-upgrade.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636.universal.universal.zip
mid-jre.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636.windows.x86-64.zip
mid-core.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636.universal.universal.zip

2) Next figure out the full URL for those files.

Since Geneva, the filename is made up of:
<Upgrade file>.<MID Buildstamp>.<platform>.<architecture>.zip
and the URLs also take a common form.
https://install.service-now.com/glide/distribution/builds/package/<file type>/<MID Buildstamp date>/<File name>

In my example the filenames are rearranged to become these URLs:

Filename: mid-upgrade.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636 .universal.universal.zip
URL: https://install.service-now.com/glide/distribution/builds/package/mid-upgrade/2019/07/16/mid-upgrade.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636.universal.universal.zip

Filename: mid-jre.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636 .windows.x86-64.zip
URL: https://install.service-now.com/glide/distribution/builds/package/mid-jre/2019/07/16/mid-jre.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636.windows.x86-64.zip

Filename: mid-core.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636 .universal.universal.zip
URL: https://install.service-now.com/glide/distribution/builds/package/mid-core/2019/07/16/mid-core.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636.universal.universal.zip

The attached Excel Spreadsheet will help with this. Just paste the MID Buildstamp from your /stats.do page in and the URL will be generated:

3) Download these files manually on another computer that has internet access and then copy those ZIP files to the MID Server folder:

<install folder>\agent\package\incoming

Note: The file names and URLs will be specific to a particular version of the platform. In this example it is Kingston Patch 1, but the same process applies to all versions.

4) Upgrade the MID Server

At this point you can wait for the AutoUpgrade thread to run again (it is on a 1 hour interval) or restart the MID Server service to force it to upgrade now.

The next time the AutoUpgrade thread runs, Downloaded: shows the files present in the <install folder>\agent\logs\agent0.log.0 log. It then goes ahead and does the upgrade.

StartupSequencer   Downloaded: [mid-upgrade.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636.universal.universal.zip, mid-core.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636.universal.universal.zip, mid-jre.newyork-06-26-2019__patch0-hotfix2-07-10-2019_07-16-2019_1636 .windows.x86-64.zip]

You can then re-use the downloaded files for any other MID Servers that are also connecting to the same instance.