Run prunsrv.exe to install the Realm Server as a Windows service. 

Example:

prunsrv.exe //IS/tibrealmserver --DisplayName="TIBCO FTL Realm Server" --Install=c:\tibco\ftl\5.0\bin\prunsrv.exe --StartMode=java --StopMode=java --StartClass=com.tibco.ftl.realmserver.RealmServer --StopClass=com.tibco.ftl.admin.RSAdmin --StartParams=--http;realmhost:10050 --StopParams=-rs;http://szou-z100:10050;-x --JavaHome="C:\Program Files\Java\jdk1.8.0_91" --LibraryPath=c:\tibco\ftl\5.0\bin  --Classpath=c:\tibco\ftl\5.0\bin\realmserver.jar;c:\tibco\ftl\5.0\bin\jetty9-all.jar;c:\tibco\ftl\5.0\bin\h2-1.3.159.jar;c:\tibco\ftl\5.0\lib\json_simple-1.1.jar;c:\tibco\ftl\5.0\lib\tibftl.jar

All parameter/values of the service are stored in the Windows registry under:

HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Apache Software Foundation\ProcRun 2.0\tibrealmserver

HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\tibrealmserver

Also, refer to FTL Administrator -> FTL processes as Windows services -> Installing the Realm server as Windows service.