Notifications

148 views

Overview


Occasionally you may need to get the MID Server thread dump or heap dump in order to troubleshoot or debug.

 

When Needed


Most common occasions where we may need this is when the MID Server is UP but having one of these issues (most common):

  • There is a latency in picking up the jobs from the instance
  • The MID Server is up but not picking any jobs from instance at all
  • Troubleshooting ITOM issues

Possible Options


We can get the thread dump from the MID Server just by logging into the instance itself.

Option 1 - Get the thread dump from the instance UI


From the instance UI we have two options:

  1. If you are a maint user:
    1. Goto MID Server record as below and make sure the MID is UP and running: Application Navigator > MID Server > Servers.
    2. Select the MID record and open the form.
    3. Click the UI action "Get thread dump"
  2. If you are an admin user you will not have access to that UI Action, so we need to run a script
    1. Go to Application Navigator > System Definition > Scripts - Background

      Copy and run this script:

      GetThreadDump();

      function GetThreadDump() {
           var mid_name = "MyMIDServer"; // replace it with the name of the mid server
           var midmanage = new MIDServerManage();
           midmanage.threaddump(mid_name);
      }

      After running the script we will get a confirmation message like this or similar:

      Background message, type:info, message: Getting MID Server thread dump

  3. Right after creating the second step, it will trigger the threaddump command on the MID Sever. You can verify this in ECC Queue.
  4. After we receiving the threaddump input in the ECC Queue it means that the thread dump is taken and is logged in the MID Server logs (Wrapper and then agent)
  5. So, to get the actual logs we need to grab the MID Server logs as below:
    Follow the step#1 and make sure the MID is up and running and on the MID Server record form you will see the UI action Grab MID logs. Click on this UI action and that will take you to ECC. 
  6. In the above step wait for the system to get the logs (Here, you need to sort it z-a to see latest ones and also try refreshing the list if no input came for the request). Basically you will see input ECC as shown in the image.
  7. Once you got the input ECC you can open the input ECC with name as wrapper.log and download the payload attachment (if any) OR get the logs within payload box.
  8. You may also want to grab the agent logs as it will have some more information.

Option 2 - Get the thread/heap dump from the MID host)


This procedure is described in KB0656229 - Get thread dump and heap dump for MID Server

 

Sample thread dump logs:


2018/05/05 13:12:05 | "StatusMonitor.600" #58 daemon prio=5 os_prio=0 tid=0x44bf8000 nid=0x14d8 in Object.wait() [0x467ef000]

2018/05/05 13:12:05 | "LogStatusMonitor.60" #57 daemon prio=5 os_prio=0 tid=0x44bf7000 nid=0x218 in Object.wait() [0x4679f000]

2018/05/05 13:12:05 | "ECCSender.1" #56 daemon prio=5 os_prio=0 tid=0x44be9c00 nid=0x12c4 in Object.wait() [0x4674f000]

Article Information

Last Updated:2018-11-03 14:52:59
Published:2018-11-03