Resolution:
TIBCO ActiveMatrix BusinessWorks Plug-in for sFTP internally uses the Jsch API. The Jsch API error with regards to the host key in wrong format seems correct as the SSH hostkeys always remains in standard format. Jsch parses all existing keys for different hosts in a known_host file and if any entry is in the wrong format, the Jsch API interprets this file as corrupted/compromised and dishonors all existing keys (though in correct format).

To resolve this issue, make sure all existing keys are in the correct format. Otherwise, use the option "Setup Host Public Key" available with the SFTP Connection shared resource which will generate a new known_host file  with the correct key. Whenever the server changes its public host key, the key content gets changed and not the format. While doing the "Setup Host Public Key", the old key gets replaced with new host key which will allow the client to connect with the server.