Logfile check sending alert from old time stamp

[ericssonvietnam]

Below is my logfile configuration file which i am using but i am receiving alert of older timestamp as logfile on server is not updating properly
Below are the log and my observation can you suggest on the same
[nagios@HNRBT1 conf]$ cat wrapperstop.conf
$seekfilesdir = '/home/nagios/logfile_basedir/tmp';
$protocolsdir = '/home/nagios/logfile_basedir/tmp';
$scriptpath = '/home/nagios/bin';
#$MACROS = { CL_DISK01 => "/dev/dsk/c0d1", CL_DISK02 => "/dev/dsk/c0d2" };

@searches = (
{
tag => 'MRTB_Jboss_wrapperstop',
criticalpatterns => ['Wrapper Stopped'] ,
okpatterns => ['"Server startup'] ,
logfile => '/opt/mrbt/jboss/server/all/log/wrapper.log'
#rotation => 'SUSE',
},
########MRBT_tomcat
{
tag => 'MRBT_Tomcat__wrapperstop',
criticalpatterns => ['Wrapper Stopped'] ,
okpatterns => ['"Server startup'] ,
logfile => '/opt/mrbt/tomcat/logs/wrapper.log'
}

);
+++++++++++++++++++++++++++++++++++++++++++
[nagios@HNRBT1 conf]$ ls -ltr /opt/mrbt/jboss/server/all/log/wrapper.log
-rw-r--r-- 1 root root 5316021 Feb 12 07:30 /opt/mrbt/jboss/server/all/log/wrapper.log
[nagios@HNRBT1 conf]$ date
Mon Feb 12 09:32:37 ICT 2018
[nagios@HNRBT1 conf]$ grep -i "Wrapper Stopped" /opt/mrbt/jboss/server/all/log/wrapper.log
STATUS | wrapper | 2017/05/16 01:08:33 | <-- Wrapper Stopped
STATUS | wrapper | 2017/05/26 01:12:21 | <-- Wrapper Stopped
STATUS | wrapper | 2017/06/15 01:01:30 | <-- Wrapper Stopped
STATUS | wrapper | 2017/08/08 01:03:15 | <-- Wrapper Stopped
STATUS | wrapper | 2017/08/08 03:29:24 | <-- Wrapper Stopped
STATUS | wrapper | 2017/08/11 02:12:35 | <-- Wrapper Stopped
STATUS | wrapper | 2017/08/25 04:06:38 | <-- Wrapper Stopped
STATUS | wrapper | 2017/10/16 23:56:20 | <-- Wrapper Stopped
STATUS | wrapper | 2017/10/17 14:50:29 | <-- Wrapper Stopped
STATUS | wrapper | 2017/10/31 04:53:11 | <-- Wrapper Stopped
STATUS | wrapper | 2018/01/16 00:38:13 | <-- Wrapper Stopped
STATUS | wrapper | 2018/01/16 00:52:45 | <-- Wrapper Stopped
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[nagios@HNRBT1 conf]$ ls -ltr /opt/mrbt/jboss/server/all/log/wrapper.log
-rw-r--r-- 1 root root 5316021 Feb 12 07:30 /opt/mrbt/jboss/server/all/log/wrapper.log
[nagios@HNRBT1 conf]$ date
Mon Feb 12 09:39:15 ICT 2018
[nagios@HNRBT1 conf]$ grep -i "Wrapper Stopped" /opt/mrbt/tomcat/logs/wrapper.log
STATUS | wrapper | 2016/08/24 01:44:16 | <-- Wrapper Stopped
STATUS | wrapper | 2016/08/24 02:26:58 | <-- Wrapper Stopped
[nagios@HNRBT1 conf]$ ls -ltr /opt/mrbt/tomcat/logs/wrapper.log
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Alert i attached for the same

Can you suggest what we can do in this case

[cdienger]

The first time check_logfile is run, it will search the entire file and note the end of the log in the seek file. Subsequent searches will start at the offset set in the seek file(/home/nagios/logfile_basedir/tmp contains the seek files). So just running the check again should clear this critical message.

From the command's documenation page at https://labs.consol.de/nagios/check_logfiles/index.html:

In principle check_logfiles scans a log file until the end-of-file is reached. The offset will then be saved in a so-called seekfile. The next time check_logfiles runs, this offset will be used as the starting position inside the log file. In the event that a rotation has occurred in the meantime, the rest of the rotated archive will be scanned also.

[ericssonvietnam]

The first time check_logfile is run, it will search the entire file and note the end of the log in the seek file. Subsequent searches will start at the offset set in the seek file(/home/nagios/logfile_basedir/tmp contains the seek files). So just running the check again should clear this critical message.

From the command's documenation page at https://labs.consol.de/nagios/check_logfiles/index.html:

In principle check_logfiles scans a log file until the end-of-file is reached. The offset will then be saved in a so-called seekfile. The next time check_logfiles runs, this offset will be used as the starting position inside the log file. In the event that a rotation has occurred in the meantime, the rest of the rotated archive will be scanned also.

Hi Cdinger
i understood the working of the plugin but still my concern is we haven't changed anything with conf file or plugin but we still get these type of alert from different nodes in certain period of time can you suggest if we can fix this out anyhow.

[cdienger]

There may be problems writing or reading the seek file. Do you see a seek file in '/home/nagios/logfile_basedir/tmp'; ? Does it get updated with offset information when the plugin is run? Is the file maybe getting deleted?

[ericssonvietnam]

There may be problems writing or reading the seek file. Do you see a seek file in '/home/nagios/logfile_basedir/tmp'; ? Does it get updated with offset information when the plugin is run? Is the file maybe getting deleted?

Again i have observed the similar issue where the alarm persist for time period longer than 30 min which is defined in the logfile conf

-bash-3.00$ cat logfile_basedir/conf/WSMS_Major.conf
$seekfilesdir = '/export/home/nagios/logfile_basedir/tmp';
$protocolsdir = '/export/home/nagios/logfile_basedir/tmp';
$scriptpath = '/export/home/nagios/bin';
#$MACROS = { CL_DISK01 => "/dev/dsk/c0d1", CL_DISK02 => "/dev/dsk/c0d2" };

@searches = (
{
tag => 'WSMS_Major',
criticalpatterns => ['MAJOR'] ,
options => 'sticky=3600,noprotocol,report=long,allyoucaneat',
logfile => '/export/home/projadm/bin/SYSTEM.LOG'

}
);

I think there is some issue with plugin or something as i am getting these types of issue from long time.

[cdienger]

options => 'sticky=3600,noprotocol,report=long,allyoucaneat',

The above would set the sticky value to an hour. To set it to a half hour use:

options => 'sticky=1800,noprotocol,report=long,allyoucaneat',