Performance Graphs suddenly stopped populating

This support forum board is for support questions relating to Nagios XI, our flagship commercial network monitoring solution.
Locked
alanjoseph
Posts: 9
Joined: Mon Jun 10, 2019 12:07 am

Performance Graphs suddenly stopped populating

Post by alanjoseph »

Hello,

We are in about day 24 of our Nagios XI evaluation, and unfortunately it stopped populating the graphs out of nowhere yesterday.
Running 5.6.3 via VMWare OVA on CentOS 7.6.1810 (part of the XI provided ova file). System Profile is attached.

I've searched through the forums and tried about everything that has been posted to no avail.
We went from a beautiful real time dashboard depicting our Core Network utilization and auto generated reports:
nagios core report.PNG
To this ... :cry:
Nagios no perf graph.PNG
I get the same results for any performance graphs across our 5 hosts and 5,000+ services.

Any ideas? I am on a time crunch, and was supposed to do a presentation on Monday showing off Nagios XI capabilities. I am worried I may need to start over from scratch.

Would changing the root password on the server cause this? As I started hardening the box to open up the email capabilities, I changed the root password, and disabled SSH from external sources. Those were the only two changes made. I also installed netstat via yum so that i could confirm no bad guys were connecting on open ports. All pretty standard stuff, so I wouldnt expect any of those to cause issues internal to Nagios.

Thanks in advance for the help! This is one of the best forums I've ever experienced.

Aloha,

Joe
You do not have the required permissions to view the files attached to this post.
User avatar
tgriep
Madmin
Posts: 9177
Joined: Thu Oct 30, 2014 9:02 am

Re: Performance Graphs suddenly stopped populating

Post by tgriep »

Is it just the Bandwidth graphs failing to populate?

If so, can you run the following commands as root and post the the /tmp/mrtg.txt file here?

Code: Select all

LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg -debug=cfg,base,log &> /tmp/mrtg.txt
LANG=C LC_ALL=C /usr/bin/mrtg &>> /tmp/mrtg.txt
LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg --lock-file /var/lib/mrtg/mrtg.lock --confcache-file /var/lib/mrtg/mrtg.ok --user=nagios --group=nagios &>> /tmp/mrtg.txt
{ time LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg 2>1 ; } 2>> /tmp/mrtg.txt

If it is all of the graphs, restart the following daemon to see if that fixes the issue.

Code: Select all

service npcd restart
Let the system run for 20 to 30 minutes and if the graphs still do not work, post the following files so we can view them.

Code: Select all

/usr/local/nagios/var/perfdata.log
/usr/local/nagios/var/npcd.log
Be sure to check out our Knowledgebase for helpful articles and solutions!
alanjoseph
Posts: 9
Joined: Mon Jun 10, 2019 12:07 am

Re: Performance Graphs suddenly stopped populating

Post by alanjoseph »

Hi There,

Yes, it appears that only the Performance Graphs are failing to populate. The Host RTT graphs appear to be fine, though that doesnt really provide what we are after for per interface stats.

I tried restarting npcd last week as part of my initial troubleshooting and no errors were apparent upon restart.

I went ahead and restarted it again, waited 30 minutes and still no performance graph data. Here are the results of the mrtg.txt, perfdata and npcd log files as requested. The mrtg.txt output was 34MB so I had to break it up into two separate files.

Sincererly appreciate the insight and feedback!

Joe
You do not have the required permissions to view the files attached to this post.
User avatar
tgriep
Madmin
Posts: 9177
Joined: Thu Oct 30, 2014 9:02 am

Re: Performance Graphs suddenly stopped populating

Post by tgriep »

I did not find any errors in the MRTG files so let's try and run the plugin command on the server as the nagios user and verbose to see if there are any errors.

Login to a shell to the server and run this to switch to the nagios user

Code: Select all

su - nagios
Run this to see what the verbose output of the plugin shows. Put in a valid IP address for Y.Y.Y.21

Code: Select all

/usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/Y.Y.Y.21_436379648.rrd -w 5.00,5.00 -c 8.00,8.00 -l G -v
Post the output as well as the output from this command ran as root.

Code: Select all

ps -ef --cols=300
Be sure to check out our Knowledgebase for helpful articles and solutions!
alanjoseph
Posts: 9
Joined: Mon Jun 10, 2019 12:07 am

Re: Performance Graphs suddenly stopped populating

Post by alanjoseph »

Here is the output. I checked another .rrd file for the trunk interface which I know should be reporting a massive amount of traffic, though it is showing 0 as well.

Code: Select all

OK - Current BW in: 0Gbps Out: 0Gbps|in=0Gb/s;5.00;8.00 out=0Gb/s;5.00;8.00
[nagios@localhost ~]$ ps -ef --cols=300
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 Jun27 ?        00:08:36 /usr/lib/systemd/systemd --switched-root --system --deserialize 22
root         2     0  0 Jun27 ?        00:00:00 [kthreadd]
root         3     2  0 Jun27 ?        00:00:04 [ksoftirqd/0]
root         5     2  0 Jun27 ?        00:00:00 [kworker/0:0H]
root         7     2  0 Jun27 ?        00:01:25 [migration/0]
root         8     2  0 Jun27 ?        00:00:00 [rcu_bh]
root         9     2  0 Jun27 ?        00:08:19 [rcu_sched]
root        10     2  0 Jun27 ?        00:00:00 [lru-add-drain]
root        11     2  0 Jun27 ?        00:00:02 [watchdog/0]
root        12     2  0 Jun27 ?        00:00:02 [watchdog/1]
root        13     2  0 Jun27 ?        00:01:21 [migration/1]
root        14     2  0 Jun27 ?        00:00:02 [ksoftirqd/1]
root        16     2  0 Jun27 ?        00:00:00 [kworker/1:0H]
root        17     2  0 Jun27 ?        00:00:02 [watchdog/2]
root        18     2  0 Jun27 ?        00:01:24 [migration/2]
root        19     2  0 Jun27 ?        00:00:03 [ksoftirqd/2]
root        21     2  0 Jun27 ?        00:00:00 [kworker/2:0H]
root        22     2  0 Jun27 ?        00:00:02 [watchdog/3]
root        23     2  0 Jun27 ?        00:01:23 [migration/3]
root        24     2  0 Jun27 ?        00:00:06 [ksoftirqd/3]
root        26     2  0 Jun27 ?        00:00:00 [kworker/3:0H]
root        28     2  0 Jun27 ?        00:00:00 [kdevtmpfs]
root        29     2  0 Jun27 ?        00:00:00 [netns]
root        30     2  0 Jun27 ?        00:00:00 [khungtaskd]
root        31     2  0 Jun27 ?        00:00:00 [writeback]
root        32     2  0 Jun27 ?        00:00:00 [kintegrityd]
root        33     2  0 Jun27 ?        00:00:00 [bioset]
root        34     2  0 Jun27 ?        00:00:00 [bioset]
root        35     2  0 Jun27 ?        00:00:00 [bioset]
root        36     2  0 Jun27 ?        00:00:00 [kblockd]
root        37     2  0 Jun27 ?        00:00:00 [md]
root        38     2  0 Jun27 ?        00:00:00 [edac-poller]
root        39     2  0 Jun27 ?        00:00:00 [watchdogd]
root        45     2  0 Jun27 ?        00:00:00 [kswapd0]
root        46     2  0 Jun27 ?        00:00:00 [ksmd]
root        47     2  0 Jun27 ?        00:01:24 [khugepaged]
root        48     2  0 Jun27 ?        00:00:00 [crypto]
root        56     2  0 Jun27 ?        00:00:00 [kthrotld]
root        58     2  0 Jun27 ?        00:00:00 [kmpath_rdacd]
root        59     2  0 Jun27 ?        00:00:00 [kaluad]
root        62     2  0 Jun27 ?        00:00:00 [kpsmoused]
root        64     2  0 Jun27 ?        00:00:00 [ipv6_addrconf]
root        77     2  0 Jun27 ?        00:00:00 [deferwq]
root       112     2  0 Jun27 ?        00:00:06 [kauditd]
apache     519  8031  0 Jul01 ?        00:00:47 /usr/sbin/httpd -DFOREGROUND
root      1625     2  0 14:33 ?        00:00:00 [kworker/0:2]
root      1941  8017  0 Jul01 ?        00:00:00 sshd: root@pts/1
root      1982  1941  0 Jul01 pts/1    00:00:00 -bash
root      2348  7416  0 15:28 ?        00:00:00 /usr/sbin/CROND -n
root      2349  7416  0 15:28 ?        00:00:00 /usr/sbin/CROND -n
root      2351  7416  0 15:28 ?        00:00:00 /usr/sbin/CROND -n
root      2352  7416  0 15:28 ?        00:00:00 /usr/sbin/CROND -n
root      2353  7416  0 15:28 ?        00:00:00 /usr/sbin/CROND -n
nagios    2359  2348  0 15:28 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1
nagios    2365  2349  0 15:28 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1
nagios    2367  2353  0 15:28 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1
nagios    2368  2351  0 15:28 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1
nagios    2369  2365  0 15:28 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php
nagios    2370  2352  0 15:28 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1
nagios    2371  2359  0 15:28 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php
nagios    2372  2367  0 15:28 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php
nagios    2373  2370  0 15:28 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php
nagios    2375  2368  0 15:28 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php
root      2393     2  0 15:28 ?        00:00:00 [kworker/3:0]
root      2527     2  0 Jun27 ?        00:00:00 [mpt_poll_0]
root      2544     2  0 Jun27 ?        00:00:00 [mpt/0]
root      2581     2  0 Jun27 ?        00:00:00 [ata_sff]
root      2638     2  0 Jun27 ?        00:00:00 [scsi_eh_0]
root      2678     2  0 Jun27 ?        00:00:00 [scsi_tmf_0]
root      2743     2  0 Jun27 ?        00:00:00 [scsi_eh_1]
root      2745     2  0 Jun27 ?        00:00:00 [scsi_tmf_1]
root      2753     2  0 Jun27 ?        00:00:00 [scsi_eh_2]
root      2757     2  0 Jun27 ?        00:00:00 [scsi_tmf_2]
root      2762     2  0 Jun27 ?        00:00:00 [ttm_swap]
root      2768     2  0 Jun27 ?        00:00:00 [irq/16-vmwgfx]
root      3186 11550  0 Jul01 ?        00:00:00 /usr/libexec/openssh/sftp-server
apache    3347  8031  0 Jul01 ?        00:00:53 /usr/sbin/httpd -DFOREGROUND
root      3655     2  0 Jun27 ?        00:00:00 [kworker/0:1H]
root      3695     2  0 Jun27 ?        00:00:00 [kdmflush]
root      3697     2  0 Jun27 ?        00:00:00 [bioset]
root      3709     2  0 Jun27 ?        00:00:00 [kdmflush]
root      3711     2  0 Jun27 ?        00:00:00 [bioset]
root      3717     2  0 14:54 ?        00:00:00 [kworker/1:2]
root      3756     2  0 Jun27 ?        00:00:50 [jbd2/dm-0-8]
root      3757     2  0 Jun27 ?        00:00:00 [ext4-rsv-conver]
root      3828     1  0 Jun27 ?        00:03:02 /usr/lib/systemd/systemd-journald
root      3854     1  0 Jun27 ?        00:00:00 /usr/sbin/lvmetad -f
root      3861     1  0 Jun27 ?        00:00:00 /usr/lib/systemd/systemd-udevd
root      5205     2  0 15:08 ?        00:00:00 [kworker/3:1]
apache    5501  8031  0 Jun30 ?        00:02:15 /usr/sbin/httpd -DFOREGROUND
nagios    5666     1  0 Jun28 ?        00:00:00 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg -f
root      6494     2  0 14:34 ?        00:00:00 [kworker/u64:1]
root      6591     2  0 15:22 ?        00:00:00 [kworker/3:3]
nagios    7039 22750  0 15:28 pts/0    00:00:00 ps -ef --cols=300
root      7074     2  0 Jun27 ?        00:00:00 [jbd2/sda1-8]
root      7086     2  0 Jun27 ?        00:00:00 [ext4-rsv-conver]
root      7381     1  0 Jun27 ?        00:00:32 /sbin/auditd
root      7405     1  0 Jun27 ?        00:00:49 /usr/sbin/irqbalance --foreground
dbus      7408     1  0 Jun27 ?        00:07:04 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation
polkitd   7414     1  0 Jun27 ?        00:02:10 /usr/lib/polkit-1/polkitd --no-debug
root      7415     1  0 Jun27 ?        00:03:01 /usr/lib/systemd/systemd-logind
root      7416     1  0 Jun27 ?        00:00:19 /usr/sbin/crond -n
chrony    7440     1  0 Jun27 ?        00:01:38 /usr/sbin/chronyd
root      7455     1  0 Jun27 ?        00:01:13 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid
root      7456     2  0 Jun27 ?        00:00:00 [kworker/1:1H]
root      7457     2  0 Jun27 ?        00:00:01 [kworker/3:1H]
root      7458     1  0 Jun27 ?        00:01:08 /usr/sbin/NetworkManager --no-daemon
root      8016     1  0 Jun27 ?        00:00:10 /usr/sbin/snmptrapd -Ln -f
root      8017     1  0 Jun27 ?        00:00:01 /usr/sbin/sshd -D
root      8018     1  0 Jun27 ?        00:00:58 /usr/bin/python2 -Es /usr/sbin/tuned -l -P
root      8024     1  0 Jun27 ?        00:01:27 /usr/sbin/rsyslogd -n
root      8031     1  0 Jun27 ?        00:00:23 /usr/sbin/httpd -DFOREGROUND
shellin+  8032     1  0 Jun27 ?        00:00:00 /usr/sbin/shellinaboxd -u shellinabox -g shellinabox --cert=/var/lib/shellinabox --port=7878 --disable-ssl-menu -s /:SSH --localhost-only --css white-on-black.css
root      8038     1  0 Jun27 ?        00:00:00 /usr/sbin/xinetd -stayalive -pidfile /var/run/xinetd.pid
shellin+  8043  8032  0 Jun27 ?        00:00:00 /usr/sbin/shellinaboxd -u shellinabox -g shellinabox --cert=/var/lib/shellinabox --port=7878 --disable-ssl-menu -s /:SSH --localhost-only --css white-on-black.css
mysql     8141     1  0 Jun27 ?        00:00:00 /bin/sh /usr/bin/mysqld_safe --basedir=/usr
mysql     8476  8141  1 Jun27 ?        01:20:27 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --log-error=/var/log/mariadb/mariadb.log --pid-file=/var/run/mariadb/mariadb.pid --socket=/var/lib/mysql/mysql.sock
root      8523     1  0 Jun27 ?        00:00:06 /usr/libexec/postfix/master -w
postfix   8525  8523  0 Jun27 ?        00:00:05 qmgr -l -t unix -u
root      8566     1  0 Jun27 ?        00:00:07 /usr/bin/perl /usr/sbin/snmptt --daemon
snmptt    8567  8566  0 Jun27 ?        00:00:13 /usr/bin/perl /usr/sbin/snmptt --daemon
root      8708     2  0 Jun27 ?        00:00:00 [kworker/2:1H]
root      8728     1  0 Jun27 ?        00:00:00 login -- root
root     11318     2  0 15:23 ?        00:00:00 [kworker/2:1]
root     11550  8017  0 Jun28 ?        00:00:00 sshd: root@pts/0
root     11583 11550  0 Jun28 pts/0    00:00:00 -bash
apache   12540  8031  0 Jun28 ?        00:04:42 /usr/sbin/httpd -DFOREGROUND
postfix  13426  8523  0 13:54 ?        00:00:00 pickup -l -t unix -u
root     14748     2  0 15:10 ?        00:00:00 [kworker/2:0]
nagios   16728     1  0 Jul01 ?        00:00:06 /usr/local/nagios/bin/npcd -d -f /usr/local/nagios/etc/pnp/npcd.cfg
apache   21279  8031  0 Jul01 ?        00:00:56 /usr/sbin/httpd -DFOREGROUND
apache   21908  8031  0 Jul01 ?        00:01:00 /usr/sbin/httpd -DFOREGROUND
root     22749 11583  0 15:25 pts/0    00:00:00 su - nagios
nagios   22750 22749  0 15:25 pts/0    00:00:00 -bash
apache   24292  8031  0 Jun27 ?        00:04:49 /usr/sbin/httpd -DFOREGROUND
nagios   25033     1  0 Jun27 ?        00:00:52 /usr/local/nagios/bin/npcd -d -f /usr/local/nagios/etc/pnp/npcd.cfg
root     26608     2  0 15:26 ?        00:00:00 [kworker/u64:0]
nagios   26907     1  0 Jun30 ?        00:10:37 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios   26908 26907  0 Jun30 ?        00:00:42 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   26909 26907  0 Jun30 ?        00:00:42 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   26910 26907  0 Jun30 ?        00:00:43 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   26911 26907  0 Jun30 ?        00:00:42 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   26912 26907  0 Jun30 ?        00:00:43 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   26913 26907  0 Jun30 ?        00:00:42 /usr/local/nagios/bin/nagios --worker /usr/local/nagios/var/rw/nagios.qh
nagios   26915  5666  0 Jun30 ?        00:01:04 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg -f
nagios   26916 26915  0 Jun30 ?        00:12:49 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg -f
nagios   26940 26907  0 Jun30 ?        00:00:16 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
apache   27347  8031  0 15:26 ?        00:00:00 /usr/sbin/httpd -DFOREGROUND
root     29383     2  0 15:20 ?        00:00:00 [kworker/1:1]
root     29430  8728  0 Jun27 tty1     00:00:00 -bash
root     31161     2  0 14:46 ?        00:00:00 [kworker/0:1]
root     31456     2  0 15:20 ?        00:00:00 [kworker/u64:2]
apache   31940  8031  0 Jun27 ?        00:04:38 /usr/sbin/httpd -DFOREGROUND
apache   31984  8031  0 Jul01 ?        00:00:44 /usr/sbin/httpd -DFOREGROUND
root     32183     2  0 15:00 ?        00:00:00 [kworker/0:0]
[nagios@localhost ~]$ 
and ...

Code: Select all

[root@localhost mrtg]# su - nagios
[nagios@localhost ~]$ /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/Y.Y.Y.21_436379648.rrd -w 5.00,5.00 -c 8.00,8.00 -l G -v
Using RRD file: /var/lib/mrtg/Y.Y.Y.21_436379648.rrd
Fetching data with command: rrdtool fetch /var/lib/mrtg/Y.Y.Y.21_436379648.rrd AVERAGE -s-10minutes | grep -vi "nan"
OK - Current BW in: 0Gbps Out: 0Gbps|in=0Gb/s;5.00;8.00 out=0Gb/s;5.00;8.00
[nagios@localhost ~]$ /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/X.X.X.212_9000212.rrd -w 5.00,5.00 -c 8.00,8.00 -l G -v
Using RRD file: /var/lib/mrtg/X.X.X.212_9000212.rrd
Fetching data with command: rrdtool fetch /var/lib/mrtg/X.X.X.212_9000212.rrd AVERAGE -s-10minutes | grep -vi "nan"
OK - Current BW in: 0Gbps Out: 0Gbps|in=0Gb/s;5.00;8.00 out=0Gb/s;5.00;8.00
User avatar
tgriep
Madmin
Posts: 9177
Joined: Thu Oct 30, 2014 9:02 am

Re: Performance Graphs suddenly stopped populating

Post by tgriep »

What is the speed of the trunk interface?
Can you post one of the rrd files from the /var/lib/mrtg folder so we can dump it and view the contents?

One more thing, check the cron log

Code: Select all

/var/log/cron
to see if there are any errors when it runs the MRTG application to gather the data from the devices.
Be sure to check out our Knowledgebase for helpful articles and solutions!
alanjoseph
Posts: 9
Joined: Mon Jun 10, 2019 12:07 am

Re: Performance Graphs suddenly stopped populating

Post by alanjoseph »

When I checked the /var/log/cron file, NOW we are getting somewhere ... It looks like we are getting the following error, the first instance of which started on JUN 26, about the time I changed the NagiosXI root password. The last successful perf graph was at 17:44, and the the error appears in the cron log at 18:25, and that was the moment all of our graphs died.

Jun 26 18:25:01 localhost crond[8325]: (root) PAM ERROR (Permission denied)
Jun 26 18:25:01 localhost crond[8325]: (root) FAILED to authorize user with PAM (Permission denied)
Casa-212 Trunk dead after access.conf changes.PNG
I also modified /etc/security/access.conf to deny root access from any sources not within our internal management network. Consequently, I commented out the line which gives root access to cron. I wasn't aware of the cron / mrtg relationship and the necessity for root access to cron until reading your thread.

Are you able to provide an example of an appropriate /etc/security/access.conf file which will accomplish the following:
1. allow nagios / mrtg to do what it needs with the least necessary privilege
2. restrict all other root access not needed

The default NagiosXI .OVA template has SSH / TTY access open to the world which we need to squelch, but in a way that doesn't break Nagios reporting. I took the following approach, which didnt work:

Code: Select all

# User "root" should get access from network 192.168.201.
# This term will be evaluated by string matching.
# comment: It might be better to use network/netmask instead.
#          The same is 192.168.201.0/24 or 192.168.201.0/255.255.255.0
+:root:192.168.201.
#
# User "root" should be denied to get access from all other sources.
-:root:ALL
I am going to re-enable root access to cron by adding this to the access.conf:

Code: Select all

# User "root" should be allowed to get access via cron .. tty5 tty6.
+:root:cron crond :0 tty1 tty2 tty3 tty4 tty5 tty6
Here is the output from the /var/log/cron which clued me in on all of this ...

Code: Select all

Jun 26 18:14:01 localhost CROND[20672]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/snmptt_service_results.php >> /usr/local/nagiosxi/var/snmptt_service_results.log 2>&1)
Jun 26 18:15:01 localhost CROND[26419]: (root) CMD (LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg --lock-file /var/lib/mrtg/mrtg.lock --confcache-file /var/lib/mrtg/mrtg.ok --user=nagios --group=nagios --user=nagios --group=nagios)
Jun 26 18:15:01 localhost CROND[26421]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Jun 26 18:15:01 localhost CROND[26420]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/snmptt_service_results.php >> /usr/local/nagiosxi/var/snmptt_service_results.log 2>&1)
Jun 26 18:15:01 localhost CROND[26424]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php >> /usr/local/nagiosxi/var/deadpool.log 2>&1)
Jun 26 18:15:01 localhost CROND[26427]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Jun 26 18:15:01 localhost CROND[26422]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Jun 26 18:15:01 localhost CROND[26426]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Jun 26 18:15:01 localhost CROND[26436]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Jun 26 18:15:01 localhost CROND[26437]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Jun 26 18:15:01 localhost CROND[26432]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Jun 26 18:15:01 localhost CROND[26434]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Jun 26 18:15:01 localhost CROND[26435]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Jun 26 18:15:01 localhost CROND[26433]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/dbmaint.php >> /usr/local/nagiosxi/var/dbmaint.log 2>&1)
Jun 26 18:16:01 localhost CROND[31655]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php >> /usr/local/nagiosxi/var/deadpool.log 2>&1)
Jun 26 18:16:01 localhost CROND[31671]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Jun 26 18:16:01 localhost CROND[31672]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Jun 26 18:16:01 localhost CROND[31674]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Jun 26 18:16:01 localhost CROND[31677]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Jun 26 18:16:01 localhost CROND[31679]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Jun 26 18:16:01 localhost CROND[31680]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Jun 26 18:16:01 localhost CROND[31683]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Jun 26 18:16:01 localhost CROND[31685]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/snmptt_service_results.php >> /usr/local/nagiosxi/var/snmptt_service_results.log 2>&1)
Jun 26 18:16:01 localhost CROND[31684]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Jun 26 18:16:01 localhost CROND[31688]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Jun 26 18:17:01 localhost CROND[4760]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Jun 26 18:17:01 localhost CROND[4761]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Jun 26 18:17:01 localhost CROND[4762]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php >> /usr/local/nagiosxi/var/deadpool.log 2>&1)
Jun 26 18:17:01 localhost CROND[4766]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Jun 26 18:17:01 localhost CROND[4765]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Jun 26 18:17:01 localhost CROND[4768]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Jun 26 18:17:01 localhost CROND[4769]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Jun 26 18:17:01 localhost CROND[4774]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Jun 26 18:17:01 localhost CROND[4773]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Jun 26 18:17:01 localhost CROND[4776]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Jun 26 18:17:01 localhost CROND[4775]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/snmptt_service_results.php >> /usr/local/nagiosxi/var/snmptt_service_results.log 2>&1)
Jun 26 18:18:01 localhost CROND[9685]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/snmptt_service_results.php >> /usr/local/nagiosxi/var/snmptt_service_results.log 2>&1)
Jun 26 18:18:01 localhost CROND[9687]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Jun 26 18:18:01 localhost CROND[9688]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php >> /usr/local/nagiosxi/var/deadpool.log 2>&1)
Jun 26 18:18:01 localhost CROND[9689]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Jun 26 18:18:01 localhost CROND[9691]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Jun 26 18:18:01 localhost CROND[9692]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Jun 26 18:18:01 localhost CROND[9697]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Jun 26 18:18:01 localhost CROND[9698]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Jun 26 18:18:01 localhost CROND[9700]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Jun 26 18:18:01 localhost CROND[9703]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Jun 26 18:18:01 localhost CROND[9707]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Jun 26 18:19:01 localhost CROND[14706]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/snmptt_service_results.php >> /usr/local/nagiosxi/var/snmptt_service_results.log 2>&1)
Jun 26 18:19:01 localhost CROND[14707]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Jun 26 18:19:01 localhost CROND[14711]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php >> /usr/local/nagiosxi/var/deadpool.log 2>&1)
Jun 26 18:19:01 localhost CROND[14709]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Jun 26 18:19:01 localhost CROND[14710]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Jun 26 18:19:01 localhost CROND[14715]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Jun 26 18:19:01 localhost CROND[14716]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Jun 26 18:19:01 localhost CROND[14719]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Jun 26 18:19:01 localhost CROND[14722]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Jun 26 18:19:01 localhost CROND[14720]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Jun 26 18:19:01 localhost CROND[14723]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Jun 26 18:20:01 localhost CROND[19948]: (root) CMD (LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg --lock-file /var/lib/mrtg/mrtg.lock --confcache-file /var/lib/mrtg/mrtg.ok --user=nagios --group=nagios --user=nagios --group=nagios)
Jun 26 18:20:01 localhost CROND[19949]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Jun 26 18:20:01 localhost CROND[19951]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Jun 26 18:20:01 localhost CROND[19950]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/snmptt_service_results.php >> /usr/local/nagiosxi/var/snmptt_service_results.log 2>&1)
Jun 26 18:20:01 localhost CROND[19953]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Jun 26 18:20:01 localhost CROND[19952]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Jun 26 18:20:01 localhost CROND[19954]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Jun 26 18:20:01 localhost CROND[19955]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/dbmaint.php >> /usr/local/nagiosxi/var/dbmaint.log 2>&1)
Jun 26 18:20:01 localhost CROND[19966]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Jun 26 18:20:01 localhost CROND[19965]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Jun 26 18:20:01 localhost CROND[19961]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Jun 26 18:20:01 localhost CROND[19967]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Jun 26 18:20:01 localhost CROND[19964]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php >> /usr/local/nagiosxi/var/deadpool.log 2>&1)
Jun 26 18:20:01 localhost CROND[19968]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Jun 26 18:21:01 localhost CROND[23763]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php >> /usr/local/nagiosxi/var/deadpool.log 2>&1)
Jun 26 18:21:01 localhost CROND[23765]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/snmptt_service_results.php >> /usr/local/nagiosxi/var/snmptt_service_results.log 2>&1)
Jun 26 18:21:01 localhost CROND[23767]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Jun 26 18:21:01 localhost CROND[23769]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Jun 26 18:21:01 localhost CROND[23771]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Jun 26 18:21:02 localhost CROND[23782]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Jun 26 18:21:02 localhost CROND[23783]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Jun 26 18:21:02 localhost CROND[23780]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Jun 26 18:21:02 localhost CROND[23784]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Jun 26 18:21:02 localhost CROND[23781]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Jun 26 18:21:02 localhost CROND[23787]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Jun 26 18:22:01 localhost CROND[27357]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Jun 26 18:22:01 localhost CROND[27358]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Jun 26 18:22:01 localhost CROND[27359]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Jun 26 18:22:01 localhost CROND[27361]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Jun 26 18:22:01 localhost CROND[27366]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Jun 26 18:22:01 localhost CROND[27371]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Jun 26 18:22:01 localhost CROND[27365]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/snmptt_service_results.php >> /usr/local/nagiosxi/var/snmptt_service_results.log 2>&1)
Jun 26 18:22:01 localhost CROND[27368]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Jun 26 18:22:01 localhost CROND[27370]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Jun 26 18:22:01 localhost CROND[27372]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php >> /usr/local/nagiosxi/var/deadpool.log 2>&1)
Jun 26 18:22:01 localhost CROND[27369]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Jun 26 18:23:01 localhost CROND[31012]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php >> /usr/local/nagiosxi/var/deadpool.log 2>&1)
Jun 26 18:23:01 localhost CROND[31013]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Jun 26 18:23:01 localhost CROND[31014]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Jun 26 18:23:01 localhost CROND[31020]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Jun 26 18:23:01 localhost CROND[31021]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Jun 26 18:23:01 localhost CROND[31023]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Jun 26 18:23:01 localhost CROND[31019]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/snmptt_service_results.php >> /usr/local/nagiosxi/var/snmptt_service_results.log 2>&1)
Jun 26 18:23:01 localhost CROND[31024]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Jun 26 18:23:01 localhost CROND[31016]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Jun 26 18:23:01 localhost CROND[31022]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Jun 26 18:23:01 localhost CROND[31026]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Jun 26 18:24:01 localhost CROND[3086]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Jun 26 18:24:01 localhost CROND[3088]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/snmptt_service_results.php >> /usr/local/nagiosxi/var/snmptt_service_results.log 2>&1)
Jun 26 18:24:01 localhost CROND[3089]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Jun 26 18:24:01 localhost CROND[3091]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Jun 26 18:24:01 localhost CROND[3096]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Jun 26 18:24:01 localhost CROND[3099]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Jun 26 18:24:01 localhost CROND[3093]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Jun 26 18:24:01 localhost CROND[3098]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php >> /usr/local/nagiosxi/var/deadpool.log 2>&1)
Jun 26 18:24:01 localhost CROND[3097]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Jun 26 18:24:01 localhost CROND[3094]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Jun 26 18:24:01 localhost CROND[3100]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Jun 26 18:25:01 localhost crond[8325]: (root) PAM ERROR (Permission denied)
Jun 26 18:25:01 localhost crond[8325]: (root) FAILED to authorize user with PAM (Permission denied)
Jun 26 18:25:01 localhost CROND[8338]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/snmptt_service_results.php >> /usr/local/nagiosxi/var/snmptt_service_results.log 2>&1)
Jun 26 18:25:01 localhost CROND[8341]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php >> /usr/local/nagiosxi/var/deadpool.log 2>&1)
Jun 26 18:25:01 localhost CROND[8344]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
I don't think posting the .rrd file will be necessary, since I am pretty sure things will start working again once root has regained access to cron. Will report back with findings in approx 30 minutes.

THANK YOU so much for pointing me int he right direction!!! It would be wonderful if Nagios could publish a post-installation hardening guide to take care of basic server hardening while giving consideration to Nagios inherent services (mrtg, MySQL, etc.)
You do not have the required permissions to view the files attached to this post.
alanjoseph
Posts: 9
Joined: Mon Jun 10, 2019 12:07 am

Re: Performance Graphs suddenly stopped populating

Post by alanjoseph »

BOOM!!!
So, allowing root access to cron in the /etc/security/access.conf file did the trick.

All Performance Graphs and related Dashboards are updating now.
Fixed Core.PNG
Fixed-Dashboard.PNG
Fixed-212-Trunk.PNG
This article was particularly helpful in deducing that PAM error in the cron log.

https://www.linuxquestions.org/question ... 175492110/

Thanks again!
You do not have the required permissions to view the files attached to this post.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Performance Graphs suddenly stopped populating

Post by scottwilkerson »

Great!

Locking thread
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
Locked