How to migrate TIBCO ActiveSpaces 2.x persistence store to Ignite for a TIBCO BusinessEvents(BE) application

How to migrate TIBCO ActiveSpaces 2.x persistence store to Ignite for a TIBCO BusinessEvents(BE) application

book

Article ID: KB0071373

calendar_today

Updated On:

Products

TIBCO BusinessEvents Enterprise Edition

Description

This article describes the file based persistence store (shared nothing) migration for a BE application from ActiveSpaces (AS) 2.x as in-built Cache to Apache Ignite.

Issue/Introduction

Migrate BE inbuild Cache datastore from AS 2.x to Ignite

Environment

All Operating Systems

Resolution

The BE utility $BE_HOME/bin/be-storedeploy can be used to migrate the cached data. 

Before starting the migration review $BE_HOME/bin/be-storedeploy.tra to be sure AS_HOME installation path is set correctly. You can also find details on this cli utility in documentation link below:

https://docs.tibco.com/pub/businessevents-enterprise/6.2.2/doc/html/Developers/Schema-Definition-Commands-Options.htm

When used for persistence store migration this utility connects to the both grids, the AS 2.x metaspace to load the data and to the Ignite grid to publish migrated data. Before you attempt the migration we should make sure CDD changes are done for Ignite migration. For instance the location of the persistence store for the grid has been configured in the CDD file with Legacy AS as cache and cluster provider and Ignite as cache and cluster provider. We recommend to modify the existing datastore and use a separate folder for Ignite datastore.
Each CacheServer stores and loads the data to and from it's own folder based on member_name. A separate folder is created for each cluster member and for each space/table.

File based persistence folder structure:
<persistenceFolderFromCDD>/<clusterName>/<tableName>/<memberName>


We recommend to connect as a AS remote client to the AS metaspace - just add "?remote=true" to the discoveryURL.
The metaspace name of the AS grid configured in CDD file "Tab Cluster" -> "General" -> "Cluster Name:" 



Windows sample script to start be-storedeploy to migrate data for cluster named "defaultAS24x":

:: BE and AS installation folder
set BE_HOME=c:\tibco\be\6.2

:: AS 2.x connect details
set DISCOVERY="tcp://localhost:50000;localhost:50001?remote=true"
set METASPACE="defaultAS24x"

:: BE agent name (for Option 2 it is required that the membername must the name of an existing CacheAgent)
set ENGINENAME=cache1

:: CDD file AS Cache
set CDD_AS="c:\BEproject\testIgniteMigr\Deployments\defaultAS24x.cdd"

:: CDD file Ignite Cache
set CDD_IGNITE="c:\BEproject\BETestProject\testIgniteMigr\Deployments\defaultIgnite.cdd"

:: ear file
set EAR="c:\BEproject\testIgniteMigr.ear"

set TRA=%BE_HOME%\bin\be-engine.tra

set PATH=%AS_HOME%\lib;%PATH%
%BE_HOME%/bin/be-storedeploy --propFile %BE_HOME%/bin/be-storedeploy.tra -u cache -op migration -c %CDD_IGNITE% %EAR% -n %ENGINENAME% -discovery %DISCOVERY% -cname %METASPACE%

Linux - sample script to start be-storedeploy to migrate data:
# BE and AS installation folder
BE_HOME=/home/tibco/be/6.2

# AS 2.x connect details
DISCOVERY="tcp://localhost:50000;localhost:50001?remote=true"
METASPACE="defaultAS24x"

# BE agent name (for Option 2 it is required that the membername must the name of an existing CacheAgent)
ENGINENAME=cache1

# CDD file AS Cache
CDD_AS="/home/tibco/BETestProject/defaultAS24xLinux.cdd"

# CDD file Ignite Cache
CDD_IGNITE="/home/tibco/BETestProject/defaultIgniteLinux.cdd"

# ear file of your project
EAR="/home/tibco/BETestProject/testIgniteMigr.ear"

TRA=$BE_HOME/bin/be-engine.tra
$BE_HOME/bin/be-storedeploy --propFile $BE_HOME/bin/be-storedeploy.tra -u cache -op migration -c $CDD_IGNITE $EAR -n $ENGINENAME -discovery $DISCOVERY -cname $METASPACE


There are two options to migrate the data.

Option #1 - All CacheServers for Ignite and AS in-built Cache has been started/running already (no data in Ignite Cache).

  • Stop InferenceAgents from AS cache cluster if they were running. This option assumes only cache agents for IGNITE cluster were started/running and no processing agents (Inference) were started/running.
  • When you started all CacheServers on same machine or only a single CacheServer disable checkbox "Enable Hostaware" in CDD file of the Ignite Cluster.
  • Start CacheServers with CDD file configured for Ignite and validate the log file to confirm that all CacheServers have joined the Ignite cluster.
  • Start be-storedeploy utility with a unique member name and wait until migration finished
  • Review data to confirm all data migrated
  • Stop CacheServers with ActiveSpaces as in-built Cache
  • Start InferenceAgents with CDD file configured for Ignite

Advantage:

  • After migration all data available in Cache and another cluster restart to recover the data from persistence store is not required
  • Data distributed and stored by all connected CacheAgents  

Disadvantage:

  • Required to set backup count at least to 1 to avoid data lost when be-storedeploy left the cluster
  • Un-used data in persistence store for the member name used by be-storedeploy - if needed you can delete the data folders for that member manually for each table.
  • Required to start CacheAgents for Ignite and AS cache at same time



Option # 2 - CacheServers for AS started and be-storedeploy creates a persistence store for an CacheServer you start laterStop InferenceAgents (AS cache)

  • Start be-storedeploy utility with a member name that match the member name of one of your CacheAgents to create a valid persistence store and wait until migration finished. The member name does not include the hostname (added by BE).
  • Start the CacheAgent with the member name used by be-storedeploy utility first
  • Start all other CacheAgents with CDD file configured for Ignite
  • Review the data to confirm migrated data recovered correctly from persistence store
  • Stop CacheServers with ActiveSpaces as in-built Cache
  • Start InferenceAgents with CDD file configured for Ignite

Advantage:

  • Not required to create a separate deployment for Ignite. Once migration finished, modify the Cache provider in CDD file and restart the agents

Disadvantage:

  • A single agent stores all data of the Ignite Cache, which can cause memory issues on the machine depending on the data size
  • After migration data not available - required to start CacheAgent(s) and recover the data from persistence store