Too high StreamBase client pageSize setting blocks client connections

Too high StreamBase client pageSize setting blocks client connections

book

Article ID: KB0076764

calendar_today

Updated On:

Products Versions
TIBCO Streaming 10

Description

None of the client applications can subscribe to output streams from the TIBCO Streaming server on the StreamBase server port (the default port is 10000). The connection fails very quickly. 

In the console log we see message:
2019-11-06 18:24:11.079000+0000 [27540:Thread- ThreadPool - 52] WARN  com.streambase.sb.sbd.TupleQueue: 
Dequeuing client's memory allocation of: 3.242 kbytes, has exceeded the maximum memory allocation of: 0 bytes.  
This may be a sign that the client is hung.
This affects commands 'epadmin dequeue stream', sbc, jsbc, sbprofile, sbmonitor, and custom Java clients using the StreamBase Client API.

 

Issue/Introduction

Set pageSize to 4K and expand the pool using maxClientPages only.

Resolution

This is caused by a "com.tibco.ep.streambase.configuration.sbclientapilistener" HOCON configuration pageSize setting which is too high and when multiplied by the maxClientPages setting results in an invalid value for the size of the pagePool.

The reason to increase the size of the pagePool is to allow some additional queuing at busy times so that the client is not disconnected immediately when it cannot keep up with the server tuples per second output rate.

The correct way to increase the pagePool size is to increase the maxClientPages value (default 4096 pages) to twice or more while leaving the pageSize setting to values between 4K and 8K. Larger pageSize values may cause large memory allocation requests when the pagePool needs to grow and may impact performance.

The combined pagePool size (pageSize * maxClientPages in bytes) is not all allocated at once, but when a client connects it is initially provided one page. The potential memory use of all clients when all are queuing is the pagePool size multiplied by the number of connections (connections * pageSize * maxClientPages). For server stability this combined size should not use most of the available JVM Heap, which is also used for server internal queues and in-memory tables.

For example, here is a ten-times increased pagePool size:
        pagePool = 
		{
			pageSize = 4K
			maxClientPages = 20480
		}
This results in a combined maximum size of 80MB and does not result in a bad pagePool setting, and as long as there is available Heap memory, clients will be able to connect.