Notifications

199 views

Description

Running more than one SCOM connector on the same MID Server can create a MID Server deadlock. The MID Server does not respond, for example, you cannot grab the MID Server log file.

Symptoms: SCOM connectors stuck in "running"  and have a "Fail" status. ("running" is a field that can be added to the connector instances form.)

Example of stack trace :
2018/05/05 18:21:15 |    java.lang.Thread.State: WAITING (parking)
2018/05/05 18:21:15 | at sun.misc.Unsafe.park(Native Method)
2018/05/05 18:21:15 | - parking to wait for  <0x00000000c3695a20> (a org.codehaus.groovy.reflection.GroovyClassValuePreJava7$GroovyClassValuePreJava7Segment)
2018/05/05 18:21:15 | at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
2018/05/05 18:21:15 | at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
2018/05/05 18:21:15 | at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
2018/05/05 18:21:15 | at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
2018/05/05 18:21:15 | at org.codehaus.groovy.util.LockableObject.lock(LockableObject.java:37)
2018/05/05 18:21:15 | at org.codehaus.groovy.util.AbstractConcurrentMapBase$Segment.removeEntry(AbstractConcurrentMapBase.java:173)
2018/05/05 18:21:15 | at org.codehaus.groovy.util.ManagedConcurrentMap$Entry.finalizeReference(ManagedConcurrentMap.java:81)
2018/05/05 18:21:16 | at org.codehaus.groovy.util.ManagedConcurrentMap$EntryWithValue.finalizeReference(ManagedConcurrentMap.java:115)
2018/05/05 18:21:16 | at org.codehaus.groovy.reflection.GroovyClassValuePreJava7$EntryWithValue.finalizeReference(GroovyClassValuePreJava7.java:51)
2018/05/05 18:21:16 | at org.codehaus.groovy.util.ReferenceManager$CallBackedManager.removeStallEntries0(ReferenceManager.java:108)
2018/05/05 18:21:16 | at org.codehaus.groovy.util.ReferenceManager$CallBackedManager.removeStallEntries(ReferenceManager.java:93)
2018/05/05 18:21:16 | at org.codehaus.groovy.util.ReferenceManager$CallBackedManager.afterReferenceCreation(ReferenceManager.java:117)
2018/05/05 18:21:16 | at org.codehaus.groovy.util.ReferenceManager$1.afterReferenceCreation(ReferenceManager.java:135)
2018/05/05 18:21:16 | at org.codehaus.groovy.util.ManagedReference.<init>(ManagedReference.java:36)
2018/05/05 18:21:16 | at org.codehaus.groovy.util.ManagedReference.<init>(ManagedReference.java:40)
2018/05/05 18:21:16 | at org.codehaus.groovy.util.ManagedLinkedList$Element.<init>(ManagedLinkedList.java:40)
2018/05/05 18:21:16 | at org.codehaus.groovy.util.ManagedLinkedList.add(ManagedLinkedList.java:102)
2018/05/05 18:21:16 | at org.codehaus.groovy.reflection.ClassInfo$GlobalClassSet.add(ClassInfo.java:478)
2018/05/05 18:21:16 | - locked <0x00000000c3602e00> (a org.codehaus.groovy.util.ManagedLinkedList)
2018/05/05 18:21:16 | at org.codehaus.groovy.reflection.ClassInfo$1.computeValue(ClassInfo.java:83)
2018/05/05 18:21:16 | at org.codehaus.groovy.reflection.ClassInfo$1.computeValue(ClassInfo.java:79)
2018/05/05 18:21:16 | at org.codehaus.groovy.reflection.GroovyClassValuePreJava7$EntryWithValue.<init>(GroovyClassValuePreJava7.java:37)
2018/05/05 18:21:16 | at org.codehaus.groovy.reflection.GroovyClassValuePreJava7$GroovyClassValuePreJava7Segment.createEntry(GroovyClassValuePreJava7.java:64)
2018/05/05 18:21:16 | at org.codehaus.groovy.reflection.GroovyClassValuePreJava7$GroovyClassValuePreJava7Segment.createEntry(GroovyClassValuePreJava7.java:55)
2018/05/05 18:21:16 | at org.codehaus.groovy.util.AbstractConcurrentMap$Segment.put(AbstractConcurrentMap.java:157)
2018/05/05 18:21:16 | at org.codehaus.groovy.util.AbstractConcurrentMap$Segment.getOrPut(AbstractConcurrentMap.java:100)
2018/05/05 18:21:16 | at org.codehaus.groovy.util.AbstractConcurrentMap.getOrPut(AbstractConcurrentMap.java:38)
2018/05/05 18:21:16 | at org.codehaus.groovy.reflection.GroovyClassValuePreJava7.get(GroovyClassValuePreJava7.java:94)
2018/05/05 18:21:16 | at org.codehaus.groovy.reflection.ClassInfo.getClassInfo(ClassInfo.java:144)
2018/05/05 18:21:16 | at com.service_now.mid.probe.tpcon.test.SCOMConnector.$getStaticMetaClass(SCOMConnector.SE.BB3B.groovy)
2018/05/05 18:21:16 | at com.service_now.mid.probe.tpcon.test.SCOMConnector.<init>(SCOMConnector.SE.BB3B.groovy)
2018/05/05 18:21:16 | at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2018/05/05 18:21:16 | at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
2018/05/05 18:21:16 | at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2018/05/05 18:21:16 | at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
2018/05/05 18:21:16 | at java.lang.Class.newInstance(Class.java:442)
2018/05/05 18:21:16 | at com.service_now.mid.probe.GroovyConnectorProbe.probe(GroovyConnectorProbe.java:62)
2018/05/05 18:21:16 | at com.service_now.mid.probe.ConnectorProbe.probe(ConnectorProbe.java:31)
2018/05/05 18:21:16 | at com.service_now.mid.probe.AProbe.process(AProbe.java:84)
2018/05/05 18:21:16 | at com.service_now.mid.queue_worker.AWorker.runWorker(AWorker.java:125)
2018/05/05 18:21:16 | at com.service_now.mid.queue_worker.AWorkerThread.run(AWorkerThread.java:20)
2018/05/05 18:21:16 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
2018/05/05 18:21:16 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
2018/05/05 18:21:16 | at java.lang.Thread.run(Thread.java:748)

 

Steps to Reproduce

 

Run more then one SCOM connector on the same MID Server.

Note that the MID Server does not respond, for example, you cannot grab the MID Server log file. SCOM connectors are stuck in "running"  and have a "Fail" status. ("running" is a field that can be added to the connector instances form.)

Workaround

Add JVM parameter to the MID Server.
 
1. Stop the MID Server service by going to the folder <MID-server-agentworkspace>/conf and open the file wrapper-override.conf file.

2, Append the following entry : 
wrapper.java.additional.103=-Dgroovy.use.classvalue=true

3. Start the MID Server service.
 
4. Confirm that the property has been applied to the MID Server by using the following MID Server background script:
var System = Packages.java.lang.System;
var prop = System.getProperty("groovy.use.classvalue");
ms.log(prop);

 


Related Problem: PRB1277549

Seen In

There is no data to report.

Fixed In

Kingston Patch 7
London

Associated Community Threads

There is no data to report.

Article Information

Last Updated:2018-09-21 15:38:06
Published:2018-07-27