How to obtain log count totals on a per log source basis in TIBCO LogLogic LMI

How to obtain log count totals on a per log source basis in TIBCO LogLogic LMI

book

Article ID: KB0077206

calendar_today

Updated On:

Products Versions
TIBCO LogLogic Log Management Intelligence all versions

Description

The Log Source Status page shows received log totals for each log source, however the Log Source Status counts will reset after a machine restart or mtask stop/start. These metrics can be valuable so use this solutions in this article to get the information you need. There are two different methods provided. The first is required for older LMI versions prior to 6.1.0 or for systems that cannot use advanced search. The 2nd method utilizes the advanced search capabilities to obtain the desired information.


 

Resolution

Method 1
For appliances that cannot execute advanced searches (not available until LMI 6.1.0) or do not have advanced features enabled, a direct database query is required to obtain this information. Use the following procedure to execute the query.

1.  SSH to the appliance and log in as ”toor”.

2.  Start the default MySQL client:
$ mysql

3.  Copy and paste the following statement to execute it:
mysql> select b.deviceName, sum(a.messages) messages from logapprtrpt.statMessages15min a, logappconfig.admDevice b where a.devID=b.deviceId group by a.devID order by messages desc;

Example:

mysql> select b.deviceName, sum(a.messages) messages from logapprtrpt.statMessages15min a, logappconfig.admDevice b where a.devID=b.deviceId group by a.devID order by messages desc;
+--------------------------------+----------+
| deviceName                     | messages |
+--------------------------------+----------+
| All Syslog Sources             |   990854 |
| ::ffff:192.168.1.240_logapp    |   926243 |
| ::ffff:192.168.1.240_otherUnix |    39300 |
| 192.168.1.240_otherFileDevice  |    16081 |
| ::ffff:192.168.1.240_logu      |     8603 |
| ::ffff:127.0.0.1_logapp        |      490 |
| ::ffff:127.0.0.1_otherUnix     |       53 |
| ::1_logapp                     |        8 |
| ::1_General                    |        1 |
+--------------------------------+----------+
9 rows in set (0.22 sec)


Note: This method relies on the retention period of your statMessages tables. The retention is configured on the "Data Retention" tab under Administration->System Settings page for LX/MX appliances. For ST appliances a direct database change is required. Execute the following query to see the current statMessages retention:
mysql> select ruleID,baseTableName,retentionTime from logappconfig.cfgTableRule where basetablename like '%statMessages%';

Example:
mysql> select ruleID,baseTableName,retentionTime from logappconfig.cfgTableRule where basetablename like '%statMessages%';
+--------+-------------------+---------------+
| ruleID | baseTableName     | retentionTime |
+--------+-------------------+---------------+
|     83 | statMessages1min  |            30 |
|     84 | statMessages5min  |            60 |
|     85 | statMessages15min |           365 |
+--------+-------------------+---------------+
3 rows in set (0.00 sec)


Now execute the following query to change the retention to the desired value if you prefer a larger time frame for use with the main query of this article for gathering log source statistics.
mysql> update logappconfig.cfgTableRule set retentionTime=XX where ruleID=85;  where XX is your new value.

If you adjust the retention to be X more days than the original retention value then it will take that many days for the additional statistics to be generated. This means your query will not immediately reflect the retention change. The statMessages15min table is recommended to be used for this method just because it already possesses a 1 year retention and because of slightly better performance because it has the lower granularity compared to the statMessages1min table.


Method 2
For appliances that can execute advanced searches (not available until LMI 6.1.0) and do have advanced features enabled, a direct database query is not required to obtain this information. Instead, a much simpler advanced search query can be used.
1. Login to the LMI GUI.
2. Enter the following query and specify the timeframe you wish to target with the query then click Run:
use system | GROUP BY sys_collectIP, sys_sourceType

Example:
User-added image


3. After the search has completed you can sort by COUNT as is shown in the screenshot.

Note that these results are predicated on the retention of the event data which is configurable on the Administration->Data Retention Rules page. Because a different retention setting controls the source information for this method of obtaining message count totals and because a different mechanism is used for recording the processed messages, the results may differ from those obtained by the first method involving a direct database query.

Issue/Introduction

This article explains how to obtain log count totals on a per log source basis using LogLogic LMI.