Missing Perf Graphs

[glitchcowboy]

We've been missing performance graphs for a very long time and I'm just getting around to inquiring.

perfdata.log hasn't been touched in a while:

[root@dooku conf.d]# tail -25 /usr/local/nagios/var/perfdata.log
...
2017-01-12 07:38:54 [9655] [0] *** TIMEOUT: /usr/local/nagios/var/spool/perfdata//1484228310.perfdata.service-PID-9655 deleted
2017-01-12 07:38:54 [9655] [0] *** Timeout while processing Host: "" Service: ""
2017-01-12 07:38:54 [9655] [0] *** process_perfdata.pl terminated on signal ALRM


[root@dooku local]# grep perfdata.log nagios/etc/pnp/process_perfdata.cfg
LOG_FILE = /usr/local/nagios/var/perfdata.log


LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg
- Throws a whole slew of errors, but completes in:
[root@dooku conf.d]# time LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg >/dev/null 2>&1

real 0m24.978s
user 0m4.550s
sys 0m0.345s

And does create rrd files

[root@dooku local]# ls -ltr /var/lib/mrtg/|tail
-rw-r--r--. 1 apache apache 105312 Dec 8 06:26 172.17.42.42_124.rrd
-rw-r--r--. 1 apache apache 105312 Dec 8 06:26 172.17.42.42_123.rrd
-rw-r--r--. 1 apache apache 105312 Dec 8 06:26 172.17.42.42_127.rrd
-rw-r--r--. 1 apache apache 105312 Dec 8 06:26 172.17.42.42_126.rrd
-rwxrwxr-x. 1 apache nagios 105312 Dec 8 06:26 172.17.254.24_5.rrd
-rwxrwxr-x. 1 apache nagios 105312 Dec 8 06:26 172.17.254.24_2.rrd
-rwxrwxr-x. 1 apache nagios 105312 Dec 8 06:26 172.17.254.24_1.rrd
-rwxrwxr-x. 1 apache nagios 105312 Dec 8 06:26 172.17.254.23_2.rrd
-rwxrwxr-x. 1 apache nagios 105312 Dec 8 06:26 172.17.254.23_1.rrd
-rwxrwxr-x. 1 apache nagios 105312 Dec 8 06:26 172.17.254.23_5.rrd

[root@dooku local]# /usr/local/nagios/libexec/check_rrdtraf -f '/var/lib/mrtg/bha-router1_20.rrd' -w 1 -c 2
OK - Current BW in: 53.14bps Out: 605.96bps|in=53.144926b/s;1;2 out=605.968519b/s;1;2


[root@dooku local]# ls -l /var/lock/mrtg/
total 0
[root@dooku local]# ls -ld /var/lock/mrtg
drwxr-xr-x. 2 root root 40 Dec 8 06:20 /var/lock/mrtg

[root@dooku mrtg]# cd /etc/mrtg/conf.d/
[root@dooku conf.d]# grep :2c *


Any help would be appreciated

[kyang]

Is this for both performance graphs and bandwidth graphs?

Is NPCD and crond running?

Code: Select all

service npcd status
service crond status

Code: Select all

service npcd restart
service crond status

LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg
- Throws a whole slew of errors, but completes in:

Could you provide us the errors that are thrown?

Here are some troubleshooting docs, I'm sure you've looked at them but is it only the bandwidth graphs?

https://support.nagios.com/kb/article.php?id=9

https://support.nagios.com/kb/article/n ... hs-29.html

[glitchcowboy]

I didn't realize that bandwidth and perf were different graphs - so I'm just using bandwidth graphs.

[root@dooku webinject]# service npcd status
NPCD running (pid 10403).
[root@dooku webinject]# service crond status
Redirecting to /bin/systemctl status crond.service
● crond.service - Command Scheduler
Loaded: loaded (/usr/lib/systemd/system/crond.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2017-12-07 20:17:26 CST; 16h ago
Main PID: 801 (crond)
CGroup: /system.slice/crond.service
└─801 /usr/sbin/crond -n

Dec 07 20:17:26 dooku.apfcu.local systemd[1]: Started Command Scheduler.
Dec 07 20:17:26 dooku.apfcu.local systemd[1]: Starting Command Scheduler...
Dec 07 20:17:26 dooku.apfcu.local crond[801]: (CRON) INFO (RANDOM_DELAY will be scaled with factor 29% if used.)
Dec 07 20:17:26 dooku.apfcu.local crond[801]: (CRON) INFO (running with inotify support)


Yes I've checked both of those articles and it all looks good.

If I limit mrtg to a single config, it looks good and still doesn't build a graph

[root@dooku mrtg]# LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg --lock-file /var/lib/mrtg/mrtg.lock --confcache-file /var/lib/mrtg/mrtg.ok
[root@dooku mrtg]# grep -i include mrtg.cfg
#Include: conf.d/*.cfg
Include: conf.d/router1.cfg
[root@dooku mrtg]#

RRD gets updated:
[root@dooku mrtg]# ls -ltr /var/lib/mrtg/|tail -1
-rw-r--r--. 1 root root 105312 Dec 8 13:06 router1_374.rrd

the perfdata.log file is still old:

[root@dooku mrtg]# ls -l /usr/local/nagios/var/perfdata.log
-rw-rw-r--. 1 nagios nagios 1029 Jan 12 2017 /usr/local/nagios/var/perfdata.log

[kyang]

What version of MRTG do you have?

Code: Select all

LANG=C LC_ALL=C /usr/bin/mrtg

Are you getting errors when you don't limit to a single config?

Code: Select all

LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg

[glitchcowboy]

[root@dooku mrtg]# LANG=C LC_ALL=C /usr/bin/mrtg
Usage: mrtg <config-file>

mrtg-2.17.4 - Multi Router Traffic Grapher

Copyright 1995-2006 by Tobias Oetiker
Licensed under the Gnu GPL.

If you want to know more about this tool, you might want
to read the docs. You can find everything on the
mrtg website:

http://oss.oetiker.ch/mrtg/

A bunch of these
2017-12-08 14:09:05: WARNING: Expected a number but got 'noSuchInstance'

some of these
2017-12-08 14:09:27: ERROR: Target[brn-router1_18][_IN_] ' $target->[652]{$mode} ' did not eval into defined data

and some of these
SNMPGET Problem for ifHCInOctets.1 ifHCOutOctets.1 on string@router1:161::::2:v4only: No response from remote host "router1" at /usr/bin/../lib/mrtg2/Net_SNMP_util.pm line 594.
Net_SNMP_util::snmpget('string@router1:161::::2:v4only', 'HASH(0x21754f0)', 'ifHCInOctets.1', 'ifHCOutOctets.1') called at /usr/bin/mrtg line 2330
main::getsnmparg('HASH(0x1530920)', 'HASH(0x2179090)', 'HASH(0x14271d0)', 'HASH(0x18cc020)') called at /usr/bin/mrtg line 2510
main::readtargets('HASH(0x1530920)', 'ARRAY(0x15ba1a8)', 'HASH(0x14271d0)') called at /usr/bin/mrtg line 403
main::main called at /usr/bin/mrtg line 143


https://imgur.com/a/Lshw1

[kyang]

Could you also run this command and attach this file?

Code: Select all

LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg -debug=cfg,base &> /tmp/mrtg.txt

Are your devices using SNMPv2 only? Or some SNMPv3?

[glitchcowboy]

I won't post that file on your forum. Please email me directly. [email protected]

[dwhitfield]

Please feel free to email [email protected] or use https://support.nagios.com/tickets/