Slowness because of high memory contention when the job 'Flow Engine Event Handler' is running
Release or Environment
New York Release
You can experience slowness on the app node where the job
Flow Engine Event Handler is running because of high memory contention, due to flow(s) looping tens of thousands or hundreds of thousands of times in "for each" flow logic.
The workaround is to refactor the flow to take the steps inside the for each loop and move them into a separate subflow. Then either:
1) (pre-Orlando) Create an action with a script step that used scriptable flow api to call the subflow (sn_fd.FlowApi.executeSubflow(...)). Call this action inside your loop.
2) (Orlando and newer) Use dynamic subflow to call the subflow. Call this subflow inside your loop.
Either option will cause the subflow execution to occur in a separate context, dividing up the steps and reducing the maximum memory used at any one time. You will know it is working by looking at sys_flow_context table and seeing that a context is created for the parent flow and then subsequent contexts are created for the subflow on each iteration.
You may also see some benefit to changing the property
com.snc.process_flow.reporting.level to value to OFF which was previously suggested.
1) Documentation will give you more insight on the property
Please see this PRB as well. Most of these high memory issues can be related to this PRB1407971.