Page 3 of 4

Re: Multiple email notifications

Posted: Wed Oct 10, 2012 10:22 am
by scottwilkerson
These both look normal.

I am sure you have tried this, but if you send a regular email to this list is it duplicated?

The reason I ask, is that you have already shown that nagios is only sending one notification in the log... Can you verify on the listserv the received emails?

Re: Multiple email notifications

Posted: Thu Oct 11, 2012 3:05 am
by aap
Thanks.

Regularv emails are not dupliacted - they are only sent once. This behaviour is very odd. I am wondering if I will need to do a database repair.

The reason I say this is that the issue only happens to a few hosts. There are hosts that have multiple notification groups and still report in the correct manner. I really don't know where else to look.

Re: Multiple email notifications

Posted: Thu Oct 11, 2012 9:41 am
by scottwilkerson
It certainly wouldn't hurt...

you can run

Code: Select all

/usr/local/nagiosxi/scripts/repairmysql.sh nagios
echo "vacuum;vacuum analyse;vacuum full;"|psql nagiosxi postgres

Re: Multiple email notifications

Posted: Fri Oct 19, 2012 4:05 am
by aap
Hi,

I have ran the database repair and the additional command that you specified with the results below:

VACUUM
VACUUM
VACUUM

Unfortunately, it has not made a difference. We are still receiving multiple alerts for "some" services.

Re: Multiple email notifications

Posted: Mon Oct 22, 2012 3:07 pm
by scottwilkerson
Is it the same few hosts? Are the notifications absolutely identical?

I'm wondering if we don't somehow have these hosts duplicated.

Re: Multiple email notifications

Posted: Wed Oct 24, 2012 2:48 am
by aap
Thanks.

I am starting to see more hosts exhibiting the same behaviuor. We monitor over 8300 services so it is getting a bit frustrating.

I have manually checked the files and there is no duplication anywhere.

The issue also can be very random. In the attached, you can see an alert at 4:27 sent twice whilst the alert at 4:47 for the same service(in purple) is sent only once. Not sure how to explain this.

Any ideas?

Moderator Note: File deleted at user request

Re: Multiple email notifications

Posted: Wed Oct 24, 2012 7:07 am
by scottwilkerson
I have a feeling this could be caused by duplicate versions of a cronjob running in parrallel

Can you run the following

Code: Select all

ps -ef |grep eventman
Not sure if we can catch it happening, but worth a try...

Re: Multiple email notifications

Posted: Wed Oct 24, 2012 8:20 am
by aap
Thanks.

Please see results below:

nagios 30894 30883 0 14:19 ? 00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/eventman.log 2>&1
nagios 30896 30894 1 14:19 ? 00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php
nagios 30904 30875 0 14:19 ? 00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/eventman.log 2>&1
nagios 30909 30904 1 14:19 ? 00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php
root 31743 29808 0 14:19 pts/0 00:00:00 grep eventman

Re: Multiple email notifications

Posted: Wed Oct 24, 2012 11:25 am
by lmiltchev
I believe we are moving to the right direction... Please, run the following commands, and post the output:

Code: Select all

cat /etc/cron.d/nagiosxi
crontab -u nagios -l

Re: Multiple email notifications

Posted: Thu Oct 25, 2012 2:11 am
by aap
See both outputs below:

cat /etc/cron.d/nagiosxi
# /etc/cron.d/nagiosxi: crontab fragment for nagiosxi

# Backup MySQL & PostgreSQL Databases
0 7 * * * root /root/scripts/automysqlbackup
0 8 * * * root /root/scripts/autopostgresqlbackup

* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php > /usr/local/nagiosxi/var/sysstat.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php > /usr/local/nagiosxi/var/cmdsubsys.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/eventman.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php > /usr/local/nagiosxi/var/feedproc.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/nom.php > /usr/local/nagiosxi/var/nom.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php > /usr/local/nagiosxi/var/reportengine.log 2>&1
*/5 * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/dbmaint.php > /usr/local/nagiosxi/var/dbmaint.log 2>&1
* * * * * nagios /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
01 * * * * nagios /usr/local/nagiosxi/cron/recurringdowntime.pl > /usr/local/nagiosxi/var/recurringdowntime.log 2>&1


crontab -u nagios -l
* * * * * /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php > /usr/local/nagiosxi/var/sysstat.log 2>&1
* * * * * /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php > /usr/local/nagiosxi/var/cmdsubsys.log 2>&1
* * * * * /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/eventman.log 2>&1
* * * * * /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php > /usr/local/nagiosxi/var/feedproc.log 2>&1
* * * * * /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1
* * * * * /usr/bin/php -q /usr/local/nagiosxi/cron/nom.php > /usr/local/nagiosxi/var/nom.log 2>&1
* * * * * /usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php > /usr/local/nagiosxi/var/reportengine.log 2>&1
*/5 * * * * /usr/bin/php -q /usr/local/nagiosxi/cron/dbmaint.php > /usr/local/nagiosxi/var/dbmaint.log 2>&1
* * * * * /usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php > /usr/local/nagiosxi/var/cleaner.log 2>&1
01 * * * * /usr/local/nagiosxi/cron/recurringdowntime.pl > /usr/local/nagiosxi/var/recurringdowntime.log 2>&1