Configuring StreamBase log output for UTF-8 international characters

Configuring StreamBase log output for UTF-8 international characters

book

Article ID: KB0081030

calendar_today

Updated On:

Products Versions
TIBCO Streaming 10

Description

The default logback configuration is not UTF-8, so international characters appear as question marks (????).
How can the log output be set to use UTF-8?

Issue/Introduction

Configuration example

Resolution

Supply a replacement logback.xml configuration file in your SB 10 project's src/main/resources folder. 

It is not enough to set system property "streambase.tuple-charset"="UTF-8" which changes tuple string field value handling within the server so that string length operators work correctly for multi-byte characters. This does not affect output.

Here is a sample logback.xml configuration file which writes correct UTF-8 characters to an alternate log named "default*_utf8.log":
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <appender name="RootConsoleAppender" class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
      <charset>UTF-8</charset>
      <pattern>%d{yyyy-MM-dd HH:mm:ss.}%usecs%d{Z} [%process:%thread] %-5level %logger: %msg%n</pattern>
    </encoder>
  </appender>

  <appender name="utf8logfile" class="ch.qos.logback.core.FileAppender">
    <file>${com.tibco.ep.dtm.logging.logDirectory}/${com.tibco.ep.dtm.logging.logFileNamePrefix}_utf8.log</file>
    <encoder>  
      <charset>UTF-8</charset>
      <pattern>%d{yyyy-MM-dd HH:mm:ss.}%usecs%d{Z} [%process:%thread] %-5level %logger: %msg%n</pattern>
    </encoder>
  </appender>
  
  <root>
    <level value="info"/>
    <appender-ref ref="RootConsoleAppender" />
    <appender-ref ref="utf8logfile" />
  </root>
</configuration>

Create this logback.xml file in the src/main/resources folder of the project.

For input text "这是一个测试。" (This is a test.) this emits:
2018-08-23 11:03:38.013000-0400 [31992:AsyncLogger] INFO  Logger: 这是一个测试。
into the file default*_utf8.log.

The normal log will continue to show:
2018-08-23 11:03:38.013000-0400 [31992:AsyncLogger] INFO  Logger: ???????

Using an alternate logback configuration is described in the documentation at:
  TIBCO StreamBase Documentation > Administration Guide > StreamBase Logging > Using StreamBase Logging