Description
Running more than one SCOM connector on the same MID Server can create a MID Server deadlock. The MID Server does not respond, for example, you cannot grab the MID Server log file.
Symptoms: SCOM connectors stuck in "running" and have a "Fail" status. ("running" is a field that can be added to the connector instances form.)
Example of stack trace :
2018/05/05 18:21:15 | java.lang.Thread.State: WAITING (parking) 2018/05/05 18:21:15 | at sun.misc.Unsafe.park(Native Method) 2018/05/05 18:21:15 | - parking to wait for <0x00000000c3695a20> (a org.codehaus.groovy.reflection.GroovyClassValuePreJava7$GroovyClassValuePreJava7Segment) 2018/05/05 18:21:15 | at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) 2018/05/05 18:21:15 | at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) 2018/05/05 18:21:15 | at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870) 2018/05/05 18:21:15 | at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199) 2018/05/05 18:21:15 | at org.codehaus.groovy.util.LockableObject.lock(LockableObject.java:37) 2018/05/05 18:21:15 | at org.codehaus.groovy.util.AbstractConcurrentMapBase$Segment.removeEntry(AbstractConcurrentMapBase.java:173) 2018/05/05 18:21:15 | at org.codehaus.groovy.util.ManagedConcurrentMap$Entry.finalizeReference(ManagedConcurrentMap.java:81) 2018/05/05 18:21:16 | at org.codehaus.groovy.util.ManagedConcurrentMap$EntryWithValue.finalizeReference(ManagedConcurrentMap.java:115) 2018/05/05 18:21:16 | at org.codehaus.groovy.reflection.GroovyClassValuePreJava7$EntryWithValue.finalizeReference(GroovyClassValuePreJava7.java:51) 2018/05/05 18:21:16 | at org.codehaus.groovy.util.ReferenceManager$CallBackedManager.removeStallEntries0(ReferenceManager.java:108) 2018/05/05 18:21:16 | at org.codehaus.groovy.util.ReferenceManager$CallBackedManager.removeStallEntries(ReferenceManager.java:93) 2018/05/05 18:21:16 | at org.codehaus.groovy.util.ReferenceManager$CallBackedManager.afterReferenceCreation(ReferenceManager.java:117) 2018/05/05 18:21:16 | at org.codehaus.groovy.util.ReferenceManager$1.afterReferenceCreation(ReferenceManager.java:135) 2018/05/05 18:21:16 | at org.codehaus.groovy.util.ManagedReference.<init>(ManagedReference.java:36) 2018/05/05 18:21:16 | at org.codehaus.groovy.util.ManagedReference.<init>(ManagedReference.java:40) 2018/05/05 18:21:16 | at org.codehaus.groovy.util.ManagedLinkedList$Element.<init>(ManagedLinkedList.java:40) 2018/05/05 18:21:16 | at org.codehaus.groovy.util.ManagedLinkedList.add(ManagedLinkedList.java:102) 2018/05/05 18:21:16 | at org.codehaus.groovy.reflection.ClassInfo$GlobalClassSet.add(ClassInfo.java:478) 2018/05/05 18:21:16 | - locked <0x00000000c3602e00> (a org.codehaus.groovy.util.ManagedLinkedList) 2018/05/05 18:21:16 | at org.codehaus.groovy.reflection.ClassInfo$1.computeValue(ClassInfo.java:83) 2018/05/05 18:21:16 | at org.codehaus.groovy.reflection.ClassInfo$1.computeValue(ClassInfo.java:79) 2018/05/05 18:21:16 | at org.codehaus.groovy.reflection.GroovyClassValuePreJava7$EntryWithValue.<init>(GroovyClassValuePreJava7.java:37) 2018/05/05 18:21:16 | at org.codehaus.groovy.reflection.GroovyClassValuePreJava7$GroovyClassValuePreJava7Segment.createEntry(GroovyClassValuePreJava7.java:64) 2018/05/05 18:21:16 | at org.codehaus.groovy.reflection.GroovyClassValuePreJava7$GroovyClassValuePreJava7Segment.createEntry(GroovyClassValuePreJava7.java:55) 2018/05/05 18:21:16 | at org.codehaus.groovy.util.AbstractConcurrentMap$Segment.put(AbstractConcurrentMap.java:157) 2018/05/05 18:21:16 | at org.codehaus.groovy.util.AbstractConcurrentMap$Segment.getOrPut(AbstractConcurrentMap.java:100) 2018/05/05 18:21:16 | at org.codehaus.groovy.util.AbstractConcurrentMap.getOrPut(AbstractConcurrentMap.java:38) 2018/05/05 18:21:16 | at org.codehaus.groovy.reflection.GroovyClassValuePreJava7.get(GroovyClassValuePreJava7.java:94) 2018/05/05 18:21:16 | at org.codehaus.groovy.reflection.ClassInfo.getClassInfo(ClassInfo.java:144) 2018/05/05 18:21:16 | at com.service_now.mid.probe.tpcon.test.SCOMConnector.$getStaticMetaClass(SCOMConnector.SE.BB3B.groovy) 2018/05/05 18:21:16 | at com.service_now.mid.probe.tpcon.test.SCOMConnector.<init>(SCOMConnector.SE.BB3B.groovy) 2018/05/05 18:21:16 | at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 2018/05/05 18:21:16 | at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 2018/05/05 18:21:16 | at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 2018/05/05 18:21:16 | at java.lang.reflect.Constructor.newInstance(Constructor.java:423) 2018/05/05 18:21:16 | at java.lang.Class.newInstance(Class.java:442) 2018/05/05 18:21:16 | at com.service_now.mid.probe.GroovyConnectorProbe.probe(GroovyConnectorProbe.java:62) 2018/05/05 18:21:16 | at com.service_now.mid.probe.ConnectorProbe.probe(ConnectorProbe.java:31) 2018/05/05 18:21:16 | at com.service_now.mid.probe.AProbe.process(AProbe.java:84) 2018/05/05 18:21:16 | at com.service_now.mid.queue_worker.AWorker.runWorker(AWorker.java:125) 2018/05/05 18:21:16 | at com.service_now.mid.queue_worker.AWorkerThread.run(AWorkerThread.java:20) 2018/05/05 18:21:16 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 2018/05/05 18:21:16 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 2018/05/05 18:21:16 | at java.lang.Thread.run(Thread.java:748)
Steps to Reproduce
Run more then one SCOM connector on the same MID Server.
Note that the MID Server does not respond, for example, you cannot grab the MID Server log file. SCOM connectors are stuck in "running" and have a "Fail" status. ("running" is a field that can be added to the connector instances form.)
Note that the MID Server does not respond, for example, you cannot grab the MID Server log file. SCOM connectors are stuck in "running" and have a "Fail" status. ("running" is a field that can be added to the connector instances form.)
Workaround
Add JVM parameter to the MID Server.
1. Stop the MID Server service by going to the folder <MID-server-agentworkspace>/conf and open the file wrapper-override.conf file.
2, Append the following entry :
2, Append the following entry :
wrapper.java.additional.103=-Dgroovy.use.classvalue=true
3. Start the MID Server service.
4. Confirm that the property has been applied to the MID Server by using the following MID Server background script:
var System = Packages.java.lang.System;
var prop = System.getProperty("groovy.use.classvalue");
ms.log(prop);
Related Problem: PRB1277549