How to roll log files at regular intervals to prevent disk space issues

How to roll log files at regular intervals to prevent disk space issues

book

Article ID: KB0076110

calendar_today

Updated On:

Products Versions
TIBCO Streaming 7.x

Description

For applications that run for long periods of time or those that send a large volume of log entries, the resulting log files may become very large and cause disk space issues on the server.  

Issue/Introduction

How to roll log files at regular intervals to prevent disk space issues.

Resolution

To manage log file size, you should use a custom logging configuration to roll your logs at specified intervals.  You can roll logs by time or by size using different appenders and triggering policies in your custom configuration. In logback, you can use a TimeBasedRollingPolicy to roll by time. For example,
 
<?xml version="1.0" encoding="UTF-8"?>

<configuration scan="true">
  <appender name="RootFileAppender" 
            class="ch.qos.logback.core.rolling.RollingFileAppender">
    <append>true</append>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <fileNamePattern>lvserver-%d{yyyy-MM-dd_HH}.log.gz</fileNamePattern>
      <!-- keep 30 days' worth of history capped at 3GB total size -->
      <maxHistory>30</maxHistory>
      <totalSizeCap>3GB</totalSizeCap>
    </rollingPolicy>
    <encoder>
      <pattern>%d{yyyy-MM-dd HH:mm:ss.SSSZ} [%thread] %-5level %logger{36} - %msg%n</pattern>
    </encoder>
  </appender>
  <appender name="RootConsoleAppender" 
            class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
      <pattern>[LogBack %level] %logger{0} - %msg%n</pattern>
    </encoder>
  </appender>
  <root>
    <level value="info"/>
    <appender-ref ref="RootFileAppender"/>
    <appender-ref ref="RootConsoleAppender"/>
  </root>
</configuration>

The above configuration would roll the console log every hour as determined by the fileNamePattern. To roll by size, use a SizeBasedTriggeringPolicy. For example,
 
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <appender name="RootFileAppender" 
            class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>sbd.log</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
      <fileNamePattern>sbd.%i.log.zip</fileNamePattern>
      <minIndex>1</minIndex>
      <maxIndex>3</maxIndex>
    </rollingPolicy>

    <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
      <maxFileSize>3GB</maxFileSize>
    </triggeringPolicy>
    <encoder>
      <pattern>%d{yyyy-MM-dd HH:mm:ss.SSSZ} [%thread] %-5level %logger{36} - %msg%n</pattern>
    </encoder>
  </appender>
  <appender name="RootConsoleAppender" 
            class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
      <pattern>[LogBack %level] %logger{0} - %msg%n</pattern>
    </encoder>
  </appender>
  <root>
    <level value="info"/>
    <appender-ref ref="RootFileAppender"/>
    <appender-ref ref="RootConsoleAppender"/>
  </root>
</configuration>

The above configuration would keep a maximum of 3 rolled log files, each being no larger than 3GB in size.

As noted in the Help, set the following system property in your sbd.sbconf configuration file to ensure that your custom logging configuration is used.
 
<java-vm>
    <sysproperty name="logback.configurationFile" value="logback-roll-by-time.xml"/>
</java-vm>