TIBCO Streaming QuickFIX/J to High-Performance FIX Engine Migration Guide

TIBCO Streaming QuickFIX/J to High-Performance FIX Engine Migration Guide

book

Article ID: KB0137909

calendar_today

Updated On:

Products Versions
TIBCO Streaming 11.1 and later

Description

The QuickFIX/J engine in Streaming is deprecated starting in the Streaming 11.0.1 release as documented here:

TIBCO Streaming > Release Notes > Recent Release Notes > Streaming 11.0.1 Release Notes
https://docs.tibco.com/pub/str/latest/doc/html/dochome/sb-110001-rn-final.html#sb-110001-3-deprecated  

The QF/J engine is expected to be removed in the 11.3.0 release. It may still be used on these versions, but we advise migrating while you have time to re-qualify your market data applications prior to upgrading to a version which lacks the QF/J engine.

The TIBCO Streaming High-Performance FIX Engine is a separately licensed component of TIBCO Streaming. If you are already using adapters with the B3 UMDF or FIX/FAST protocol, then you are already using the StreamBase High-Performance FIX Engine.

The "StreamBase High-Performance FIX Engine" may be identified by the following alternative names in the product documentation:

  • "TIBCO Enterprise Streaming - High-Performance FIX Engine"
  • "TIBCO StreamBase High-Performance FIX Engine"
  • "StreamBaseFIX"
  • "High-Performance FIX"
  • "HPFIX"

Resolution

FIX Adapters

The Streaming 11.2.0 list of adapters which use a FIX engine are:

Adapter

 

FIX Adapter (generic)

360T SuperSonic TEX Adapter

Barclays BARX FIX Adapter

Bloomberg Tradebook FX FIX Adapter

Bosla Comercio Santiago FIX Adapter

CitiFX FIX Adapter

Currenex Adapter

Deutsche Bank Autobahn FX Classic FIX Adapter

Deutsche Bank Autobahn FX Rapid FIX Adapter

FXAll FIX Adapter

FXSpotStream FIX Adapter

GAIN GTX FIX Adapter

Goldman Sachs Electronic Trading FX Adapter

HSBC FIX Adapter

Integral FX Inside FIX Adapter

KCG Hotspot FX FIX Adapter

LavaFX FIX Adapter

Morgan Stanley FIX Adapter

Nomura FX FIX Adapter

Raptor Adapter

Saxo Bank FIX Adapter

Trading Technologies Adapter

UBS FIX Adapter

The FIX Adapter has built-in FIX dictionaries for FIX versions FIX.4.0 through FIX.5.0 SP2 (with FIXT.1.1). The venue specific FIX adapters also come with Antenna dictionaries for immediate use.  Use the generic FIX Adapter with any Antenna dictionary provided by unlisted vendors. Currently the HPFIX engine version used in Streaming 11.2.0 does not use QuickFIX dictionaries, but we expect this capability to be added in a future release. In the interim, TIBCO Technical Support can convert a custom QuickFIX dictionary into an Antenna dictionary at your request.

Adapter Differences

The QF/J and HPFIX engines support a slightly different set of native capabilities.

The differences are as follows:

Feature

QF/J

HPFIX

Setting custom fields on outgoing FIX messages

QFJAdminMessageInterceptor onAppMessage(), onAdminMessage()

FIXAJMessageInterceptor onAdminMessage()

User-Level Logon

Username/Password/NewPassword

<username>/<password>/<new-password>

Replay Messages Command

not available

available using the Command Input Port

logtype=slf4j/screen

available

not available

FIX Configuration File

a properties file

an XML file and properties file

Fast Auto-Acknowledge

not available

available

The HPFIX onAdminMessage adds custom fields to all sent messages, whereas the QF/J onAppMessage allows for message inspection to select which messages receive custom fields. See section below, "FIX Message Interceptors".

Basic Migration Process

Migration Overview

The architecture of the FIX adapter allows for changing the FIX engine with minimal changes to properties and EventFlow code. Most changes are to the FIX engine configuration. Since the HPFIX engine features are mostly a superset of those provided by QuickFIX, there are only a few cases where EventFlow logic must change. See the "Special Cases" section that follows.

Install the StreamBase High Performance FIX Engine package

Install the StreamBase High Performance FIX Engine package from https://www.tibco.com/downloads/10982 matching your installed TIBCO Streaming version (the choices are versions 11.1.1 or 11.2.0 at the time of this writing) on every developer and build system. The install package may also be found by logging into https://www.tibco.com/downloads and searching for the "TIBCO StreamBase High-Performance FIX Engine" or "TIBCO Enterprise Streaming - High-Performance FIX Engine".

The HPFIX installer adds to the installed Streaming product directories the files:

./sdk/maven/product-artifact/sb-hp-fix-11.2.0-hp-fix-sdk-offline-projects.zip
./sdk/maven/product-artifact/sb-hp-fix-11.2.0-hp-fix-sdk.mrcatalog
./sdk/maven/product-artifact/sb-hp-fix-11.2.0-hp-fix-sdk.zip

The feature is not fully available until the local Maven repository is populated.

To populate the local Maven repository, run from the StreamBase Command Prompt: epdev install maven

The same installer IS NOT USED on production server installations. When an application is built in a developer or build environment the necessary StreamBase High Performance FIX Engine libraries are included in the resulting application archive.

Update the Project Dependencies

Add to the project pom.xml file dependencies:

     <dependency>
        <groupId>com.tibco.ep.sb.thirdparty.com.epam</groupId>
        <artifactId>fixaj-adaptor</artifactId>
        <version>2.24.1</version>
    </dependency>

    <dependency>
        <groupId>com.tibco.ep.sb.thirdparty.com.epam</groupId>
        <artifactId>fixaj-engine-all</artifactId>
        <version>2.24.1</version>
    </dependency>
    
    <dependency>
        <groupId>jakarta.xml.bind</groupId>
        <artifactId>jakarta.xml.bind-api</artifactId>
        <version>2.3.2</version>
    </dependency>

    <dependency>
        <groupId>org.glassfish.jaxb</groupId>
        <artifactId>jaxb-runtime</artifactId>
        <version>2.3.2</version>
    </dependency> 

If you see the following "NoClassDefFoundError" message, check the pom.xml dependencies listed above. This error occurs if they are not correct:

Adapter Exception: Resume exception for FIXOutput_2 : Error creating instance of session handler using FIX engine 'Streaming High-Performance Engine'. Cause: com/epam/fixengine/ExtendedFIXSessionListener: NoClassDefFoundError: ClassNotFoundException: com.epam.fixengine.ExtendedFIXSessionListener: Adapter Exception: Resume exception for FIXOutput_2 : Error creating instance of session handler using FIX engine 'Streaming High-Performance Engine'. Cause: com/epam/fixengine/ExtendedFIXSessionListener: NoClassDefFoundError: ClassNotFoundException: com.epam.fixengine.ExtendedFIXSessionListenercom.kabira.platform.Transaction$InvocationRunException: Adapter Exception: Resume exception for FIXOutput_2 : Error creating instance of session handler using FIX engine 'Streaming High-Performance Engine'. Cause: com/epam/fixengine/ExtendedFIXSessionListener: NoClassDefFoundError: ClassNotFoundException: com.epam.fixengine.ExtendedFIXSessionListener

Update the adapter configuration

Refer to the Streaming documentation for the FIX Adapter
https://docs.tibco.com/pub/str/latest/doc/html/adaptersguide/embeddedFIX.html 

The QF/J configuration is a name-value paired properties file with sections:

  • [DEFAULT]
  • [SESSION]

The sessions are unnamed. This differs from the HPFIX configuration which is a structured XML file with the hierarchy:

  • <sessions><session name="session-name">...</session></sessions>

There is no equivalent for the QF/J "default" section, so common settings must be duplicated within each session group.

 

For detailed descriptions of all configuration options, see:

A. QuickFIX/J Configuration
https://www.quickfixj.org/usermanual/2.3.0/usage/configuration.html 
(Not all settings described here apply to the QF/J version (1.4.0) provided in Streaming. Use this reference to describe in-use features to inform conversion to HPFIX.)

B. FIXAntennaJava: Configuration
https://corp-web.b2bits.com/FAJ_doc/fixaj-manual/page15.html 

Save for later adding new FIX engine configuration options beyond those needed to replace your QF/J configuration. The HPFIX engine has additional functionality which may be explored later once you have a working configuration that preserves application function.

Common Setting Conversions

On the left is the QF/J setting and on the right is the nearest matching HPFIX setting.

QuickFIX setting

HPFIX setting

Note

ConnectionType=initiator

<initiator>true</initiator>

 

ConnectionType=acceptor

<initiator>false</initiator>

 

FileStoreMaxCachedMsgs=0

not applicable (na)

 

BeginString=FIX.4.2

<begin-string>FIX.4.2</begin-string>

The version and begin-string settings may be different for those cases where this is needed.

SenderCompID=MySenderID

<sender-comp-id>FIXAdapter_1</sender-comp-id>

 

TargetCompID=MyTargetID

<target-comp-id>FIXAdapter_2</target-comp-id>

 

SocketAcceptPort=1234

<port>1234</port>

 

FileLogPath=logs/

na

 

FileStorePath=store/

<storage-factory></storage-factory>

Typically empty for HPFIX where these fixengine.properties settings are defined:

  • com.epam.fixengine.storage.FilesystemStorageFactory
  • com.epam.fixengine.storage.InMemoryStorageFactory
 

<storage-directory></storage-directory>

Used with a FilesystemStorageFactory

SocketConnectionHost=localhost

<acceptor-hostname>localhost</acceptor-hostname>

 

SocketConnectPort=1234

<port>1234</port>

 

HeartBtInt=30

<heartbeat-interval>30</heartbeat-interval>

 

UseDataDictionary=Y

<dictionary>myFixDict</dictionary>

Set only with <dictionary-file>, otherwise empty

DataDictionary=myDictionaryFile

<dictionary-file>myDictionaryFile</dictionary-file>

Set only with <dictionary>, otherwise empty

logtype=slf4j

na

Logging to the engine log is controlled through FIX adapter StreamBase Properties for "Trace Management"

logtype=screen

na

QF/J settings Screen* have no HPFIX equivalent

ResetOnLogon=Y

<reset-on-logon>true</reset-on-logon>

 

FileIncludeMilliseconds=Y

<use-date-milliseconds>true</use-date-milliseconds>

 
 

<message-interceptor></message-interceptor>

 

Username=MyUsername
Password=MyCurrentPassword
NewPassword=MyNextPassword

<username>MyUsername</username>
<password>MyCurrentPassword</password>
<new-password>MyNextPassword</new-password>

 

SocketUseSSL=N
SocketKeyStore=file_path
SocketKeyStorePassword=pass
SocketTrustStore=file_path
SocketTrustStorePassword=pass

<use-ssl>false</use-ssl>

Set JVM java system properties
javax.net.ssl.keyStore,
javax.net.ssl.keyStorePassword,
javax.net.ssl.trustStore, etc.

StartDay=Mon
EndDay=Fri
Weekdays=Sun,Mon,Tue,...
StartTime=12:30:00
EndTime=21:30:00

<scheduled-start>30 09 * * *</scheduled-start>
`30 16 * * *`

See http://en.wikipedia.org/wiki/Cron#CRON_expression for options and syntax.

AllowedRemoteAddresses=

<acceptor-ip-whitelist>...

 
   

See additional fixengine.properties settings in the FIX Adapter documentation


Update the adapter StreamBase properties in StreamBase Studio

  1. Set  Fix Engine To Use  to "StreamBaseFIX".
  2. Set  FIX Configuration File  to the HPFIX formatted configuration file as described in the previous section.
  3. Set  FIX Admin Message Interceptor Class  to a new implementation of the Interceptor. See below section "FIX Message Interceptors".
  4. Set  FIX Application Message Interceptor Class  to a new implementation of the Interceptor. See below section "FIX Message Interceptors".

All other adapter property settings are the same for both the QF/J and HPFIX engines and no changes are expected for compatibility.

Special Cases

Custom Dictionaries

If the data provider has customized their service to use a FIX version with a modified dictionary, which is common for the Login (A) command and for additional messages to use special features of their service, then obtain their B2Bits Antenna or QuickFIX dictionary for use with the HPFIX adapter.

If a data provider does not provide a custom dictionary in this format, please obtain their FIX Specification for reference when modifying a supplied dictionary. The FIX version and specification will be needed when contacting Support for assistance with the FIX adapters in these cases.

To author your own custom FIX dictionary, start with the standard dictionary text-file for the required version and edit that to suit. The standard FIX dictionaries are supplied with Streaming and are selectable and configurable.  These include: versions/BeginString: FIX.4.0, FIX.4.1, FIX.4.2, FIX.4.3, FIX.4.4, FIX.5.0, FIX.5.0SP1, FIX.5.0SP2, and FIXT.1.1 (used with FIX.5.0 and later versions).

After Streaming product install and local Maven repository setup (using SB Studio or 'epdev'), the HPFIX dictionaries may be found in:
  C:\Users\{USER}\.m2epository\com\tibco\ep\sb\thirdparty\com\epam\fixaj-engine-all\2.24.1\fixaj-engine-all-2.24.1.jar\
Filenames: fixdic40.xml (40, 41, 42, 43, 44, 50, 50sp1, 50sp2, 50sp2ep)

The HPFIX dictionary format is (syntax example):

<fixdic>
 <typelist>
   <typedef type="int">
     <descr>
     </descr>
   </typedef>
   <typedef type="String">
     <descr>
     </descr>
   </typedef>  
</typelist>      
 <fielddic>
   <valblockdef id="Country" name="Country">
     <item val="AF">AFGHANISTAN</item>
     ...
   </valblockdef>    
   ...
  <fielddef tag="1" name="Account" type="String">
    <descr>
    </descr>
  </fielddef>    
  ...
   <fielddef tag="112" name="TestReqID" type="String">
     <descr>
     </descr>
   </fielddef>
   ...    
 </fielddic>
 <msgdic>
   <blockdef id="CommissionData" name="Commission Data">
     <field tag="12" name="Commission"/>
     ...
   </blockdef>
   <msgdef msgtype="0" name="Heartbeat" admin="Y">
     <field tag="112" name="TestReqID">
     </field>
     ...
     <descr>
     </descr>
   </msgdef>
   <msgdef msgtype="1" name="Test Request" admin="Y">
     <field tag="112" name="TestReqID" req="Y"/>
     ...
   </msgdef>    
 </msgdic>
</fixdic>


To add a field to a message, find the message definition "msgdef" section and add a "field" following the above pattern.

FIX Message Interceptors

There is an important difference between the QF/J implementation of method onAdminMessage() and the HPFIX methods getOutgoingLoginFields() and getHeaderFields().

  • The QF/J implementation intercepts the outgoing message and allows it to be inspected and modified based on specific conditions, whereas the HPFIX implementation is static and adds the additional fields to every message without inspection.
  • The QF/J implementation is divided into two interfaces:
    • com.streambase.sb.adapter.fix.engine.quickfixj.QFJAdminMessageInterceptor
    • com.streambase.sb.adapter.fix.engine.quickfixj.QFJAppMessageInterceptor
  • The HPFIX implementation has a single interface:
    • com.streambase.sb.adapter.fix.engine.antenna.FIXAJMessageInterceptor

Since the most frequent use-case for using the onAppMessage() and onAdminMessage() methods is to add custom fields and field values, a more preferred approach is to use a custom FIX dictionary which alters the affected message types individually to add optional custom fields. This exposes those fields at the StreamBase EventFlow implementation level in the tuple schema, and allows them to be set normally. Since this changes the schemas used within your EventFlow application, this change will involve adding these field values in the output tuple sent into the FIX output adapter using a Map or other adapter that sets a new output schema.

For example, the QF/J onAdminMessage() override allows the method to identify, for example, a Logon(A), Heartbeat(0), or Reject(3) message and add specific header and body fields. The equivalent HPFIX method getOutgoingLoginFields() requires all field values to be defined at startup, added to all Admin messages, and NOT be able to algorithmically determine field presence and values later in the session. Our recommendation is that this not be implemented with a new FIXAJMessageInterceptor, but set in your EventFlow application as described in the previous paragraph.

This is a current limitation with CoinBase (https://coinbase.com) which modifies the Logon(A) message (https://docs.cdp.coinbase.com/exchange/fix-api/market-data) to require a calculated RawData(96) field based on message timestamp and other fields, values which are only known to the HPFIX engine at the time the message is constructed. With no method that exposes the to-be-sent message to the adapter Java API layer this field cannot be populated. We advise that for Market Data, users make use of the CoinBase Websocket feed (https://docs.cdp.coinbase.com/exchange/websocket-feed/overview). For Order Entry, you may continue to use the CoinBase Order Entry FIX service which does not require the RawData(96) field for message authentication (https://docs.cdp.coinbase.com/exchange/fix-api/order-entry-messages/order-entry-messages5). This is TIBCO Streaming issue "SB-53061 Coinbase FIX API Adapter".

Issue/Introduction

This guide answers common configuration and usage questions for the TIBCO Streaming High-Performance FIX Engine (HPFIX) used as a replacement for the QuickFIX/J Engine (QF/J). All adapters that use the FIX protocol in TIBCO Streaming require an engine. This guide is a supplement to the TIBCO Streaming product documentation for the FIX Adapter, which may be referenced here: 

TIBCO Streaming > Adapters Guide > TIBCO Streaming Embedded Adapters > FIX Adapter
https://docs.tibco.com/pub/str/latest/doc/html/adaptersguide/embeddedFIX.html 

Additional Information

KB Articles

How to add optional FIX header fields to outgoing messages using the StreamBase High Performance FIX engine
https://support.tibco.com/support-home/kbsearch/article?articleNumber=KB0074036

Streaming FIX adapter ResendRequest and the Message Replay command
https://support.tibco.com/support-home/kbsearch/article?articleNumber=KB0074537