alerts only run once, scheduler seems not to work

[_asp_]

Hi,

I have a problem with the alerting mechanism in logserver.
If I click on "run alert now" evereything seems to be fine. I get an email if conditions are matched and the timestamp of "last run" is updated.

But then, the last run will not being updated again and i will not get any emails again.
I assume that the scheduling of alerts is somehow broken. We are on logserver 1.4.0.
Are there any logs of the logserver we can check to give you further details?

Also deactivating and activating again does not run the scheduled alerts.

If I create a new alert in status active it will not even run once, until i click on "run alert now"

Thanks, Andreas

[hsmith]

Can you try to reset the jobs?

Go to Administration > System > Command Subsystem. Click on 'Reset All Jobs'

[lrhazi]

I have the same issue! reset-all-jobs does not seem to fix it. am testing Nagios Log Server 1.4.3

where are the logs for this functionality ?

Thanks a lot,
Mohamed.

[mcapra]

I was unable to replicate this on my NLS 1.4.2 or 1.4.3 machine.

From Administration -> Audit Reports, do you see any messages that indicate the run_alerts job has failed?

[lrhazi]

I only see the manual runs I did by myself.. dont seem to see anything about the run that should be occurring every minute!

2016-11-09T15:32:11.816-05:00 ALERT Alert Name test-alert returned CRITICAL: 2 matching entries found |logs=2;1;1

[lrhazi]

I installed on CentOS 7, using this command:

curl https://assets.nagios.com/downloads/nag ... install.sh | sudo sh

Is there a cronjob that needs to be running for this? or processes I need to inspect?

It does not seem like any task is running at all, the command subsystem shows Last Run as Never :


Job ID Job Status Last Run Status Last Run Time Frequency Next Run Time Type Actions
cleanup_cmdsubsys Waiting - Never 1 hour 11/11/2016 16:31:25 System Edit
backups Waiting - Never 1 day 11/12/2016 15:31:25 System Edit
backup_maintenance Waiting - Never 1 day 11/12/2016 15:31:25 System Edit
run_all_alerts Waiting - Never 20 seconds 11/11/2016 14:31:45 System Edit
run_update_check Waiting - Never 1 day 11/11/2016 15:31:25 System Edit

[rkennedy]

Usually this is due to a problem with jobs / poller not running properly. Could you post these files for us to look at? /usr/local/nagioslogserver/var/jobs.log and /usr/local/nagioslogserver/var/poller.log

What happens if you execute these manually?

Code: Select all

/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php poller
/usr/bin/php -q /var/www/html/nagioslogserver/www/index.php jobs

[lrhazi]

Thank you. I can't find those two log files anywhere!!

➜ ~ sudo find / -name jobs.log
➜ ~ sudo find / -name poller.log
➜ ~ sudo find / -name index.php
/tmp/nagioslogserver/nagioslogserver/index.php
/tmp/nagioslogserver/nagioslogserver/basedir/html/www/index.php
/var/www/html/index.php
/var/www/html/nagioslogserver/www/index.php
➜ ~
➜ ~ sudo -unagios -H /usr/bin/php -q /var/www/html/nagioslogserver/www/index.php poller
Updating Cluster Hosts File
Updating Elasticsearch with instance...
Updating Cluster Hosts File
Updating Elasticsearch with instance...
Updating Cluster Hosts File
Updating Elasticsearch with instance...
Updating Cluster Hosts File
Updating Elasticsearch with instance...
Finished Polling.
➜ ~
➜ ~ sudo -unagios -H /usr/bin/php -q /var/www/html/nagioslogserver/www/index.php jobs
Running command run_alerts with args ' ' for job id: run_all_alerts
SUCCESS
Running command run_alerts with args ' ' for job id: run_all_alerts
SUCCESS
Running command run_alerts with args ' ' for job id: run_all_alerts
SUCCESS
Processed 0 node jobs.
Processed 3 global jobs.

[rkennedy]

They may be hanging - after you ran them manually, can you go back to look to see if the logs exist? Did your alerts update?

What is the output of these commands?

Code: Select all

grep "date.timezone" /etc/php.ini
ls -l /etc/localtime
php -r 'echo date("D M j G:i:s T Y")."\n";'
date

[lrhazi]

No, still no log files... which is odd, I see those command in a crontab... if they run at all, it should create zero byte files, right?

Code: Select all

[root@logdev01 ~]# ls -l /usr/local/nagioslogserver/var/
total 12
-rwxrwxr-x. 1 nagios nagios 24 Nov 15 08:55 cluster_hosts
-rw-rw-r--. 1 nagios nagios 37 Nov  7 09:41 cluster_uuid
-rw-rw-r--. 1 nagios nagios 37 Nov  7 09:41 node_uuid
[root@logdev01 ~]#

Code: Select all

[root@logdev01 ~]# cat  /etc/cron.d/nagioslogserver
# /etc/cron.d/nagioslogserver: crontab fragment for nagioslogserver

* * * * * nagios /usr/bin/php -q /var/www/html/nagioslogserver/www/index.php poller > /usr/local/nagioslogserver/var/poller.log 2>&1
* * * * * nagios /usr/bin/php -q /var/www/html/nagioslogserver/www/index.php jobs > /usr/local/nagioslogserver/var/jobs.log 2>&1

although, what is nagios in the beginning of those crontab commands! is "nagios" a command? or is this crontab irrelevant?

Code: Select all

[root@logdev01 ~]# grep "date.timezone" /etc/php.ini
; http://php.net/date.timezone
date.timezone = US/Eastern
[root@logdev01 ~]# ls -l /etc/localtime
lrwxrwxrwx 1 root root 30 Nov 15 08:58 /etc/localtime -> /usr/share/zoneinfo/US/Eastern
[root@logdev01 ~]# php -r 'echo date("D M j G:i:s T Y")."\n";'
Tue Nov 15 15:02:56 EST 2016
[root@logdev01 ~]# date
Tue Nov 15 15:02:59 EST 2016
[root@logdev01 ~]#