Nagios XI IO issues after upgrade...

[JakeHatMacys]

I think it's got something to do with the Ramdisk. But since upgrading i'm having all kinds of backend DB issues and I've run DB repairs and restarted everything and now i'm a bit stuck...

Capture.JPG

And the system doesn't appear to be that busy from a hardware perspective...

Capture2.JPG

If I look in perfdata.log:

2017-09-01 18:58:25 [21255] [0] *** Timeout while processing Host: "*****" Service: "_HOST_"
2017-09-01 18:58:25 [21253] [0] *** Timeout while processing Host: "******" Service: "Server_Ping_Check"
2017-09-01 18:58:25 [21255] [0] *** process_perfdata.pl terminated on signal ALRM
2017-09-01 18:58:25 [21253] [0] *** process_perfdata.pl terminated on signal ALRM
2017-09-01 18:58:25 [21257] [0] *** TIMEOUT: Timeout after 40 secs. ***
2017-09-01 18:58:25 [21257] [0] *** TIMEOUT: Deleting current file to avoid NPCD loops
2017-09-01 18:58:25 [21257] [0] *** TIMEOUT: Please check your npcd.cfg
2017-09-01 18:58:25 [21257] [0] *** TIMEOUT: /var/nagiosramdisk/spool/perfdata//1504287835.perfdata.service-PID-21257 deleted
2017-09-01 18:58:25 [21257] [0] *** Timeout while processing Host: "*******" Service: "Server_Ping_Check"
2017-09-01 18:58:25 [21257] [0] *** process_perfdata.pl terminated on signal ALRM

So it seems like something is definitely going on with the ramdisk after upgrading....

But also now I can see that my HPBSM integration daemon won't start...

[1504322868] HP BSM Integration: Could not push message to shared memory queue: queue may be full; check if the integration daemon is running!
[1504322868] HP BSM Integration: Could not push message to shared memory queue: queue may be full; check if the integration daemon is running!
[1504322868] HP BSM Integration: Could not push message to shared memory queue: queue may be full; check if the integration daemon is running!
[1504322868] HP BSM Integration: Could not push message to shared memory queue: queue may be full; check if the integration daemon is running!
[1504322868] HP BSM Integration: Could not push message to shared memory queue: queue may be full; check if the integration daemon is running!


I've restarted it several times and even looked up an similar issue where commenting out the HP BSM integration which doesn't seem to fix my issue....

[scottwilkerson]

the I/O wait is extremely high, which would lead me to believe you are having a disk issue of some sort.

Lets verify it is not full

Code: Select all

df -h
df -i

I would check to see if there are any hardware issues on the hypervisor if this is a VM.

Finally, if may be best for diagnosis if you could either PM a profile from Admin -> System Profile or open a ticket by emailing [email protected] as we will need to dig into what is causing the issue if it is not hardware related.

[tmcdonald]

I see you have also opened an email ticket for this, so we will be locking this thread and continuing in the email ticket.