Perf Graphing issues

[BanditBBS]

Having some graphing issues on my main server. There are minimum amount of graphs working, quite a bit that stopped working when I updated earlier today and quite a few that seem to have not worked in a while. I enabled logging for perfdata debug and see this quite a bit:

Code: Select all

2014-06-12 19:31:07 [15383] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/svwddprdbdb02/System_Memory.rrd: illegal attempt to update using time 1402615842 when last update time is 1402615842 (minimum one second step)
2014-06-12 19:31:07 [15383] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/svwddprdbdb01/Disk.rrd: illegal attempt to update using time 1402615842 when last update time is 1402615842 (minimum one second step)
2014-06-12 19:31:07 [15383] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/rp000001/_MQHA_FFCLSP02.rrd: illegal attempt to update using time 1402615842 when last update time is 1402615842 (minimum one second step)
2014-06-12 19:31:07 [15383] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/rp000006/__Disk_Usage.rrd: illegal attempt to update using time 1402615852 when last update time is 1402615852 (minimum one second step)
2014-06-12 19:31:07 [15383] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/rp000006/_MQHA_FFOMSP01.rrd: illegal attempt to update using time 1402615852 when last update time is 1402615852 (minimum one second step)
2014-06-12 19:31:07 [15383] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/rp000006/Memory_Usage.rrd: illegal attempt to update using time 1402615852 when last update time is 1402615852 (minimum one second step)
2014-06-12 19:31:22 [15612] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/aewddb03/_HOST_.rrd: illegal attempt to update using time 1402615862 when last update time is 1402615862 (minimum one second step)
2014-06-12 19:31:22 [15612] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/aewddb02/_HOST_.rrd: illegal attempt to update using time 1402615862 when last update time is 1402615862 (minimum one second step)
2014-06-12 19:31:22 [15612] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/svwddprdbdb01/_HOST_.rrd: illegal attempt to update using time 1402615862 when last update time is 1402615862 (minimum one second step)
2014-06-12 19:31:22 [15612] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/WDCAE-PTEA01B.AEO.AE.COM/_HOST_.rrd: illegal attempt to update using time 1402615862 when last update time is 1402615862 (minimum one second step)
2014-06-12 19:31:22 [15613] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/rp000007/__Disk_Usage.rrd: illegal attempt to update using time 1402615862 when last update time is 1402615862 (minimum one second step)
2014-06-12 19:31:22 [15613] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/rp000006/Load.rrd: illegal attempt to update using time 1402615862 when last update time is 1402615862 (minimum one second step)
2014-06-12 19:31:22 [15613] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/rp000007/_MQHA_FFOMSP02.rrd: illegal attempt to update using time 1402615862 when last update time is 1402615862 (minimum one second step)
2014-06-12 19:31:22 [15613] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/rp000006/__Disk_Usage.rrd: illegal attempt to update using time 1402615862 when last update time is 1402615862 (minimum one second step)
2014-06-12 19:31:22 [15613] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/rp000006/_MQHA_FFOMSP01.rrd: illegal attempt to update using time 1402615862 when last update time is 1402615862 (minimum one second step)
2014-06-12 19:31:22 [15613] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/rp000006/Memory_Usage.rrd: illegal attempt to update using time 1402615862 when last update time is 1402615862 (minimum one second step)
2014-06-12 19:31:22 [15612] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/svwddprdbdb03/_HOST_.rrd: illegal attempt to update using time 1402615862 when last update time is 1402615862 (minimum one second step)
2014-06-12 19:31:22 [15613] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/rp000002/__Disk_Usage.rrd: illegal attempt to update using time 1402615862 when last update time is 1402615862 (minimum one second step)
2014-06-12 19:31:22 [15613] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/WDCAE-PTEA03B.AEO.AE.COM/Drive_C__Disk_Usage.rrd: illegal attempt to update using time 1402615862 when last update time is 1402615862 (minimum one second step)
2014-06-12 19:31:22 [15613] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/WDCAE-PTEA02B.AEO.AE.COM/System_Log_Critical_Errors.rrd: illegal attempt to update using time 1402615862 when last update time is 1402615862 (minimum one second step)
2014-06-12 19:31:22 [15613] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/WDCAE-PTEA02B.AEO.AE.COM/Drive_C__Disk_Usage.rrd: illegal attempt to update using time 1402615862 when last update time is 1402615862 (minimum one second step)
2014-06-12 19:31:22 [15613] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/WDCAE-PTEA02B.AEO.AE.COM/Application_Log_Warnings.rrd: illegal attempt to update using time 1402615862 when last update time is 1402615862 (minimum one second step)
2014-06-12 19:31:22 [15613] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/WDCAE-PTEA02B.AEO.AE.COM/Page_File_Usage.rrd: illegal attempt to update using time 1402615862 when last update time is 1402615862 (minimum one second step)
2014-06-12 19:31:22 [15613] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/WDCAE-PTEA01B.AEO.AE.COM/Drive_C__Disk_Usage.rrd: illegal attempt to update using time 1402615862 when last update time is 1402615862 (minimum one second step)
2014-06-12 19:31:22 [15613] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/svwddprdbdb02/System_Memory.rrd: illegal attempt to update using time 1402615862 when last update time is 1402615862 (minimum one second step)
2014-06-12 19:31:22 [15613] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/svwddprdbdb01/Disk.rrd: illegal attempt to update using time 1402615862 when last update time is 1402615862 (minimum one second step)
2014-06-12 19:31:22 [15613] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/rp000001/_MQHA_FFCLSP02.rrd: illegal attempt to update using time 1402615862 when last update time is 1402615862 (minimum one second step)
2014-06-12 19:31:37 [15836] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/rp000007/Swap_Usage.rrd: illegal attempt to update using time 1402615882 when last update time is 1402615882 (minimum one second step)
2014-06-12 19:31:37 [15836] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/rp000007/__Disk_Usage.rrd: illegal attempt to update using time 1402615882 when last update time is 1402615882 (minimum one second step)
2014-06-12 19:31:37 [15836] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/rp000006/Load.rrd: illegal attempt to update using time 1402615882 when last update time is 1402615882 (minimum one second step)
2014-06-12 19:31:37 [15836] [1] RRDs::update ERROR /usr/local/nagios/share/perfdata/rp000007/_MQHA_FFOMSP02.rrd: illegal attempt to update using time 1402615882 when last update time is 1402615882 (minimum one second step)

Any ideas?

[Box293]

Let's check the system time to start off with.

Goto Admin > System Profile.

What is shown under Date/Time? For example:

Code: Select all

Date/Time
PHP Timezone: Australia/Sydney
PHP Time: Fri, 13 Jun 2014 11:16:05 +1000
System Time: Fri, 13 Jun 2014 11:16:05 +1000

[BanditBBS]

Code: Select all

Date/Time
PHP Timezone: America/New_York 
 PHP Time: Thu, 12 Jun 2014 21:21:14 -0400
 System Time: Thu, 12 Jun 2014 21:21:14 -0400

[Box293]

Well that all looks good.

From what it looks like, I think the RRD files are trying to be updated multiple times for the same time period.

I'm not sure where to go next with this, we'll have to wait until tommorrow when the Nagios team wakes up. Sorry I can't be of any more help right now.

[BanditBBS]

I figured out why it isn't graphing anymore. Now I just need help figuring out why it is doing what it is doing.

All my checks are being returned with a "\n" on the end and that is screwing up the perf data. This is only happening on stuff that goes out to gearman. Localhost and passive results do not have this issue.

Anyone have any clue, anyone else with gearman having this issue as well?

[slansing]

Ah, the infamous newline character issue, this was fixed in 2014 r1.1, or was supposed to be. If you took a look at my short guide to getting mod_gearman working with Core 4, there is a manual hack/fix for it as well-

Page 2: http://support.nagios.com/forum/viewtop ... 16&t=25594

[BanditBBS]

I changed the two commands and applied changes and I am still getting the new line in performance data.....hmmmm

EDIT: DANGIT, it needs to point to the ramdisk in that command, now to figure out what that needs to be, lol.
EDIT2: Fixed and fixed. Lock this up!