Resolution: When you run multiple IM engines, they are fault tolerant i.e if one engine goes down or crashes, then the other engine takes over. By default IM uses a dedicated rv daemon for the FT heartbeats. The RV daemon used for FT can be configured using the following properties:
When running RV/RVCM job creators(JC) in a multi-engine setup, these JCs will be active in only one engine. This kind of setup will primarily act as a fault tolerant setup. But if you are using checkpointing feature and if load increases on the active engine, you will see some jobs migrating to the other engine. You will see partial load balancing happening.
If you wish to make this setup(RV/RVCM JCs) strictly fault tolerant, then set the following properties to false: Config.Primary.JobPool.JobMigration Config.Primary.JobPool.JobCreatorMigration
These two properties, when set to false, will turn off IMs internal load balancing.
If you plan to run this fault tolerant setup under extreme load, we recommend setting the following property to true: Config.Primary.JobPool.FT.BackupFTCheck
Under extreme load, the RV daemon may not get enough CPU cycles to send out FT hearbeats. This could cause false triggers, due to which the inactive engine may get activated and start processing jobs of the other active engine. This could lead to duplicate processing and erratic behaviour. The 'BackupFTCheck' property helps in this kind of scenario.