Description

• Mid servers are down with the below errors on the logs

StartupSequencer SEVERE *** ERROR *** Problem invoking InstanceInfo on https://<instance_name>.service-now.com/: Please check that the InstanceInfo page exists in the sys_public table and active="true".   StartupSequencer SEVERE *** ERROR *** Field instance_name missing in InstanceInfo response   StartupSequencer SEVERE *** ERROR *** test failure   java.lang.IllegalStateException: Unable to connect to instance.   at com.service_now.mid.services.StartupSequencer.runTests(StartupSequencer.java:455)   at com.service_now.mid.services.StartupSequencer$Starter.run(StartupSequencer.java:416)   StartupSequencer Successfully connected to instance:   StartupSequencer Install name: Demo Server   StartupSequencer Instance name: nwhealthconnection   StartupSequencer Node: 4334c5b5fccba97a72858860b6b60ff1   StartupSequencer Build date: 03-02-2020_2243   StartupSequencer Build tag: glide-newyork-06-26-2019__patch7-02-25-2020   StartupSequencer Instance ID: 563d716ddb7faa404f035d00cf9619c6   StartupSequencer System ID: app132144.sjc201.service-now.com:nwhealthconnection009   StartupSequencer Instance IP: 10.90.132.144   StartupSequencer MID buildstamp: newyork-06-26-2019__patch7-02-25-2020_03-02-2020_2243   StartupSequencer WARNING *** WARNING *** Invalid response to MIDServerCheck query   StartupSequencer SEVERE *** ERROR *** test failure  java.lang.IllegalStateException: User cannot be authenticated or is missing the proper roles. If you have deleted or changed the MID server keystore, and config.xml mid.instance.password value is encrypted, you may need to change this value to plain text (during MID  startup, password is re-encrypted using current keystore and written back to mid.instance.password).  at com.service_now.mid.services.StartupSequencer.runTests(StartupSequencer.java:462)  at com.service_now.mid.services.StartupSequencer$Starter.run(StartupSequencer.java:416)

Release or Environment

• Newyork and below

Resolution

• The very first step is to check from the "sys_public" table if the page "InstanceInfo" exists and its value is "true".
  
  
• The next step is to check from the "sys_web_service" table if the below listed are set to "true".
  • InstanceInfo
  • GetMIDInfo
  • InstanceInfo
  • MIDAssignedPackages
  • MIDFieldForFileProvider
  • MIDFileSyncSnapshot
  • MIDServerCheck
  • MIDServerFileProvider
    
    
• Observe for the below errors from the "wrapper" logs
  
  

  INFO | jvm 6 | yyyy/mm/dd 00:35:55.983 | mmap failed for CEN and END part of zip file  INFO | jvm 6 | yyyy/mm/dd 00:35:55.983 | mmap failed for CEN and END part of zip file  INFO | jvm 6 | yyyy/mm/dd 00:35:55.983 | mmap failed for CEN and END part of zip file  INFO | jvm 6 | yyyy/mm/dd 00:35:55.983 | mmap failed for CEN and END part of zip file  INFO | jvm 6 | yyyy/mm/dd 00:35:55.983 | mmap failed for CEN and END part of zip file  INFO | jvm 6 | yyyy/mm/dd 00:35:55.983 | mmap failed for CEN and END part of zip file  INFO | jvm 6 | yyyy/mm/dd 00:35:55.983 | mmap failed for CEN and END part of zip file  INFO | jvm 6 | yyyy/mm/dd 00:35:55.983 | mmap failed for CEN and END part of zip file  INFO | jvm 6 | yyyy/mm/dd 00:35:59.193 | mmap failed for CEN and END part of zip file  INFO | jvm 6 | yyyy/mm/dd 00:35:59.194 | mmap failed for CEN and END part of zip file  INFO | jvm 6 | yyyy/mm/dd 00:35:59.194 | mmap failed for CEN and END part of zip file  INFO | jvm 6 | yyyy/mm/dd 00:35:59.194 | mmap failed for CEN and END part of zip file  INFO | jvm 6 | yyyy/mm/dd 00:36:00.297 | mmap failed for CEN and END part of zip file  INFO | jvm 6 | yyyy/mm/dd 00:36:00.297 | mmap failed for CEN and END part of zip file

• If the above errors are found, the solution is to restart the application nodes.
  
  
• Restarting the application nodes should help providing the relief.
  
  
• This issue is fixed in Orlando, hence a suggestion is to upgrade the instance to "Orlando".