Page 2 of 3
Re: Missing RRD graphs
Posted: Wed Dec 23, 2015 3:38 pm
by Box293
CFT6Server wrote:So does Nagios create a job somewhere to check and populate the graphs or does it just pick up anything that ends up in the folder? I have a feeling whatever mechanism that is, that's where it went south. So the wizard created the service checks and MRTG config, but the Nagios side of things isn't picking it up or not knowing that it needs to process this particular perfdata?
It's the check command that looks at the rrd file
check_xi_service_mrtgtraf!server4_2.rrd!7,7!8,8!G.
Can you please run this command and send us the output:
It's all very odd.
Re: Missing RRD graphs
Posted: Wed Dec 23, 2015 3:42 pm
by CFT6Server
It is a bit odd....
# ps -ef | grep nagios.cfg
root 9456 13948 0 12:42 pts/4 00:00:00 grep nagios.cfg
nagios 17569 1 6 11:54 ? 00:02:51 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios 17642 17569 0 11:54 ? 00:00:00 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
Re: Missing RRD graphs
Posted: Wed Dec 23, 2015 3:44 pm
by Box293
Do you see any errors when you run the following command?
Code: Select all
LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg
Re: Missing RRD graphs
Posted: Wed Dec 23, 2015 3:52 pm
by CFT6Server
Just warnings on some checks that don't exist, but not relating to these server. Since the rrd file creates in /var/lib/mrtg, I don't suspect there are any issues on the MRTG side of things. Just really not sure why it doesn't pick it up on the Nagios side. Does Nagios keep a database of what to check for? or it just picks up whatever is in a folder when check for performance data?
Re: Missing RRD graphs
Posted: Wed Dec 23, 2015 4:00 pm
by Box293
CFT6Server wrote:Just really not sure why it doesn't pick it up on the Nagios side. Does Nagios keep a database of what to check for? or it just picks up whatever is in a folder when check for performance data?
It just looks at that specific file. You can test at the command line, it would be:
Code: Select all
su nagios
/usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/server2_2.rrd -w 7,7 -c 8,8 -l G
You do have a check interval of 15 minutes, so does forcing an immediate check of the service work?
Re: Missing RRD graphs
Posted: Wed Dec 23, 2015 4:10 pm
by CFT6Server
Forcing immediate checks does not yield anything different.
Here's something interesting though....
$ /usr/local/nagios/libexec/check_rrdtraf -f /var/lib/mrtg/server2_2.rrd -w 7,7 -c 8,8 -l G
OK - Current BW in: 0Gbps Out: 0Gbps|in=.000045Gb/s;7;8 out=.000179Gb/s;7;8
and forcing a check immediately just keep giving /var/lib/mrtg/server2_2.rrd does not exist
No perfdata though in the nagios folder...
Code: Select all
# ls -l
total 7172
-rwxrwxr-x 1 nagios nagios 1151496 Dec 23 12:38 CPULoad.rrd
-rwxrwxr-x 1 nagios nagios 3450 Dec 23 12:38 CPULoad.xml
-rwxrwxr-x 1 nagios nagios 384952 Dec 23 12:41 CPU.rrd
-rwxrwxr-x 1 nagios nagios 2178 Dec 23 12:41 CPU.xml
-rwxrwxr-x 1 nagios nagios 384952 Dec 23 12:49 DiskAll.rrd
-rwxrwxr-x 1 nagios nagios 2224 Dec 23 12:49 DiskAll.xml
-rwxrwxr-x 1 nagios nagios 1918040 Dec 23 12:56 DiskIO_sda.rrd
-rwxrwxr-x 1 nagios nagios 4827 Dec 23 12:56 DiskIO_sda.xml
-rwxrwxr-x 1 nagios nagios 1534768 Dec 23 13:05 _HOST_.rrd
-rw-rw-r-- 1 nagios nagios 3968 Dec 23 13:05 _HOST_.xml
-rwxrwxr-x 1 nagios nagios 384952 Dec 23 12:56 RAM.rrd
-rwxrwxr-x 1 nagios nagios 2276 Dec 23 12:56 RAM.xml
-rwxrwxr-x 1 nagios nagios 1151496 Dec 23 12:45 SwapIO.rrd
-rwxrwxr-x 1 nagios nagios 3483 Dec 23 12:45 SwapIO.xml
-rwxrwxr-x 1 nagios nagios 384952 Dec 23 12:44 Swap.rrd
-rwxrwxr-x 1 nagios nagios 2238 Dec 23 12:44 Swap.xml
# pwd
/usr/local/nagios/share/perfdata/server2
Re: Missing RRD graphs
Posted: Wed Dec 23, 2015 4:12 pm
by Box293
Can you try:
Go into CCM
Tools > Write Config Files
Click the Delete button
It will say "Successfully deleted all Host / Service Config Files"
Click the Write Button
It will show an output of all the files it creates
Quick Tools > Apply Configuration
Click the Apply Configuration button
Now go and force an immediate check.
Re: Missing RRD graphs
Posted: Wed Dec 23, 2015 4:29 pm
by CFT6Server
So re-writing the configurations doesn't help. Still getting the same issue.
I think at this point, I am not sure what else we can do, I know re-creating the service check will fix our issue. Perhaps a dev at Nagios can chime in on what else could be wrong.
I might leave one service and just redo the configuration for the rest.
Re: Missing RRD graphs
Posted: Wed Dec 23, 2015 4:41 pm
by CFT6Server
Hmm interesting... I just removed the checks on "server5" and recreated it. Now it is showing the same error message....
I am noticing that some of the devices I added yesterday shows this issue... seems to be random devices out of a list of about 20.
Updated: I've removed the MRTG files /var/lib/mrtg/ files and service checks, then it came back fine....
Re: Missing RRD graphs
Posted: Wed Dec 23, 2015 5:10 pm
by rkennedy
So at this point did the RRD graphs come back?