474 views

Description

App nodes are rendered off line with "Too Many Open Files" error messages. Cause of the issue appears to be linked to unterminated FTP connections.

Example:

main,Thread-6956,4
java.net.PlainSocketImpl.socketAccept(Native Method)
java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
java.net.ServerSocket.implAccept(ServerSocket.java:462)
java.net.ServerSocket.accept(ServerSocket.java:430)
org.ftp4che.FTPConnection.sendPortCommand(FTPConnection.java:858)
org.ftp4che.FTPConnection.streamFile(FTPConnection.java:1086)
org.ftp4che.FTPConnection.access$0(FTPConnection.java:1062)
org.ftp4che.FTPConnection$1$DownStreamingThread.run(FTPConnection.java:1035)


Looking in the logs we can see a number of failed attempts at downloading an FTP file, example:

----------------
java.io.IOException: /glide/nodes/pret004_16519/bin/../tmp/gurle219a17951f21100c974a5160812dc60.tmp size:0

Connection properties:
user.password=**********
connection.passive=false
connection.timeout=10000
user.login=ServiceNow
connection.port=21
connection.type=FTP_CONNECTION
connection.host=ftp2.pret.com
----------------------
331 Password required for servicenow
230 Logged on
211-Features:
MDTM
REST STREAM
SIZE
MLST type*;size*;modify*;
MLSD
UTF8
CLNT
MFMT
211 End
215 UNIX emulated by FileZilla
200 Type set to I
200 Port command successful
150 Opening data channel for file transfer.
( scheduled import job: APIResolvedFTP)
-------------------------------------

Supposition: We are failing to clean up the connections after a bad connection attempt - over time these grow and eventually we hit the O/S limit and ergo the node effectively fails

Steps to Reproduce

 

Workaround

There is no workaround for this Known Error at this time.

Related Problem: PRB595004

Seen In

Berlin Patch 11
Calgary Patch 1
Calgary Patch 2 Hot Fix 5
Calgary Patch 3 Hot Fix 1
Calgary Patch 5

Fixed In

Calgary Patch 5
Dublin EA 9
Eureka

Associated Community Threads

There is no data to report.

Article Information

Last Updated:2016-03-23 08:37:27
Published:2014-03-31