In order to generate a JCo trace at Design time, the JCo trace VM arguement should be set in the TIBCOBusinessStudio.ini under the "-vmargs" section. The change would take effect after user restarts BusinessStuido. The file "TIBCOBusinessStudio.ini" is in the %TIB_HOME%/studio/<version>/eclipse folder.

The following is an example of TIBCOBusinessStudio.ini that could output a JCo trace at Design time.

-startup
../../../eclipse-platform/bundlepool/1.0/org.eclipse.equinox.p2.touchpoint.eclipse/plugins/org.eclipse.equinox.launcher_1.3.0.v20140415-2008.jar
--launcher.library
../../../eclipse-platform/bundlepool/1.0/org.eclipse.equinox.p2.touchpoint.eclipse/plugins/org.eclipse.equinox.launcher.win32.win32.x86_64_1.1.200.v20140603-1326
-showlocation
-showsplash
../../../eclipse-platform/bundlepool/1.0/org.eclipse.equinox.p2.touchpoint.eclipse/plugins/com.tibco.bw.branding.bwe.design
--launcher.XXMaxPermSize
256m
-vm
c:/tibco/bw635/tibcojre64/1.8.0/bin/server/jvm.dll
-install
c:/tibco/bw635/studio/4.0/eclipse
-vmargs
-Dfile.encoding=UTF-8
-Dorg.osgi.framework.bootdelegation=javax.xml.*
-Dorg.xml.sax.driver=com.sun.org.apache.xerces.internal.parsers.SAXParser
-Djco.trace_level=8
-Djco.trace_path=c:/log/
-Djrfc.trace=1
-XX:MaxPermSize=256m
-XX:+UnlockDiagnosticVMOptions
-XX:+UnsyncloadClass
-XX:+UseParNewGC
-Xms512m
-Xmx2048m

How to generate a JCo trace for working with BusinessStudio in Design time.