Repeated jgroups tcp errors in cs_server.log, including "does not match own cookie"

Repeated jgroups tcp errors in cs_server.log, including "does not match own cookie"

book

Article ID: KB0076284

calendar_today

Updated On:

Products Versions
TIBCO Data Virtualization All supported versions

Description


Scenario:  TIBCO Data Virtualization (TDV) instance's cs_server.log file contains frequent entries resembling the text below:


WARN [ConnectionTable.Acceptor,null,null] 2020-01-30 09:36:29.365 -0800 TCPConnectionMap - Could not accept connection from peer 
WARN [ConnectionTable.Acceptor,null,null] 2020-01-30 09:36:29.365 -0800 TCPConnectionMap - Could not accept connection from peer 
java.net.SocketException: ConnectionTable.Connection.readPeerAddress(): cookie read by 10.96.28.145:9407 does not match own cookie; terminating connection 
at org.jgroups.blocks.TCPConnectionMap$TCPConnection.readPeerAddress(TCPConnectionMap.java:489) 
at org.jgroups.blocks.TCPConnectionMap$TCPConnection.<init>(TCPConnectionMap.java:375) 
at org.jgroups.blocks.TCPConnectionMap$ConnectionAcceptor.run(TCPConnectionMap.java:252) 
at java.lang.Thread.run(Thread.java:748)

Issue/Introduction

This concerns unusually frequent logging in the cs_server.log.

Resolution

The TDV base port +7 (ie 9407) is reserved for cluster communications between cluster nodes. If there is an outside agent that is contacting this port, you will see these messages logged.

This logging can be manually reproduced by using a SQL jdbc/odbc client that mistakenly points to our base port +7  (ie; 9407) instead of the correct base port +1 (ie; 9401).
Every connection attempt towards this base port +7 will log these jgroups errors.

Look for any agent that is erroneously contacting your cluster port.  It could be something such as a misconfigured client tool or load balancer.