The Swagger framework used to test the REST service is accessed using the port specified by the bw.rest.docApi.port property in the  "<TIBCO_HOME>\bw\<version>\domains\<domain name>\appnodes\<appspace name>\<appnode name>\config.ini " file. By default, this property is set to port 7777 & it specifies the port number on which Swagger framework serves the API's,documentation endpoint, e.t.c.

The feature can be disabled using BW_DISABLE_SWAGGER_UI property.

There is a hidden property to disable swagger doc. You can adding the following to VM arguments (for studio design time) or java.extended.properties (in appnode TRA for runtime)  to disable the same

DesignTime(VM arguments section under Run/Debug Configuration => Arguments tab)
===========

-Dbw.appnode=BWEclipseAppNode -Xms256m -Xmx2048m -Dorg.eclipse.emf.ecore.EPackage.Registry.INSTANCE=com.tibco.tpcl.org.eclipse.emf.OSGiEnabledPackageRegistry -Dosgi.bundles.defaultStartLevel=5 -Dosgi.compatibility.bootdelegation=true -Dosgi.contextClassLoaderParent=ext -Dosgi.parentClassloader=ext -Dorg.osgi.framework.system.packages=org.osgi.framework -Dorg.osgi.framework.startlevel.beginning=10 -Dosgi.classloader.copy.natives=true -Dosgi.classloader.type=parallel -Dosgi.bundlefile.limit=20 -Declipse.stateSaveDelayInterval=10000 -Declipse.ignoreApp=true -Dosgi.noShutdown=true -Dorg.osgi.service.http.port=8090 -DTIBCO_HOME="C:/tibco/bw6" -DBW_HOME="C:/tibco/bw6/bw/6.3" -Dbw.governance.enabled=true -DBW_DISABLE_SWAGGER_UI=true

Runtime:
=========
java.extended.properties=-Xmx1024m -Xms128m -XX:+HeapDumpOnOutOfMemoryError -DBW_DISABLE_SWAGGER_UI=true


You may see a warning post adding this property in appnode log(extract shown below) that is safe to ignore.
=======
WARN  [qtp1145512119-91] com.tibco.thor.frwk.Application - Failed to obtain Swagger URL information for tibco.bw.sample.binding.rest.Basic.application
java.lang.IllegalArgumentException: Swagger UI is disabled
​=======