Resolution:
Description:
===========
TIBCO BusinessEvents takes a long time to startup and throws the JDBC error: Error while creating database connection

Environment:
===========
TIBCO BusinessEvents 5.x
Operating System: Linux
Oracle Database 11g

Symptoms:
=========
TIBCO BusinessEvents takes significant amount of time to start up (10min +) and throws a JDBC connection error.Here is an example of log:
++++++++++++
Info [main] - [jdbcstore] Initializing DB connection pool: schema=Default schema, key=/SharedResources/Connections/JDBC/JDBC Connection.sharedjdbc, initialSize=10, minSize=10, maxSize=10
Info [main] - [jdbcstore] EnforcePoolSize=false, WaitTimeout=1, InactivityTimeout=900
Warning [main] - [jdbcstore.impl] Making connection failed : Io exception: Connection reset
Warning [main] - [jdbcstore.impl] Failed to create/recover connections to database: jdbc:oracle:thin:@xxxxx:xxx:xxx
Warning [main] - [jdbcstore.impl] Error - Io exception: Connection reset
Fatal [main] - [jdbcstore] JdbcStore initialization failed.
java.lang.Exception: Error while creating database connection
        at com.tibco.be.jdbcstore.impl.DBConnectionPool.init(SourceFile:125)
        at com.tibco.be.jdbcstore.impl.DBConnectionPoolManager.init(SourceFile:30)
++++++++++++

Cause:
=========
Users upgrading to Oracle 11g can encounter this issue if the underlying OS is Linux which is running on a faulty hardware.

More details on the cause:

java.security.SecureRandom is a standard API provided by SUN. Among various methods offered by this class void nextBytes(byte[]) is one. This method is used for generating random bytes. Oracle 11g JDBC drivers use this API to generate random number during login. Users using Linux have been encountering SQLException("Io exception: Connection reset").

The problem is two fold

1). The JVM tries to list all the files in the /tmp (or alternate tmp directory set by -Djava.io.tmpdir) when SecureRandom.nextBytes(byte[]) is invoked. If the number of files is large the method takes a long time to respond and causes the server to timeout

2). The method void nextBytes(byte[]) uses /dev/random on Linux and on some machines which lack the random number generating hardware the operation slows down to the extent of bringing the whole login process to a halt.

Ultimately, the user encounters SQLException("Io exception: Connection reset")
Users upgrading to 11g can encounter this issue if the underlying OS is Linux which is running on faulty hardware. The cause of this has not yet been determined. It could either be a problem with the hardware or the fact that the software cannot read from dev/random

Resolution:
=========
In the tra file, add -Djava.security.egd=file:/dev/../dev/urandom to java.extended.properties.


Reference:
=========
https://kr.forums.oracle.com/forums/thread.jspa?messageID=3699989
http://www.usn-it.de/index.php/2009/02/20/oracle-11g-jdbc-driver-hangs-blocked-by-devrandom-entropy-pool-empty

TIBCO BusinessEvents takes a long time to startup and throws the JDBC error: Error while creating database connection