Naigosxi preformance issue

[strmon]

Hi Support,

Need your help in troubleshooting NagiosXI performance. Please let me know what details you need from system from understanding.

My configuration are as follows:
Server:
OS: centos 6 64 bit
Mem: 12 GB
CPU : 6 (vcpu)

NagiosXI: 5.4.3
host in monitoring: 98
Services : 1980


Do let me know what other details you need from my side to troubleshoot the issue.

[scottwilkerson]

Can you describe the problem you are experiencing (be as detailed as possible)?

If we can get a detailed description of the problem, as well as have you PM myself or another tech you system profile zip from
Admin -> System Profile

That would be a good start

[strmon]

Now and then we receive error like,
Performance Grapher : not running
Database Maintenance : delay

Also, system become close to freeze and its hard to even ssh. We recently upgraded system memory from 8 to 12 Gb, still no luck.

In order to troubleshoot, i've preformed steps shared in Nagiosxi performance guide, after that i found it bit better bit still not preformance graph is not working.

As requested, i've attached profile zip, have a look and suggest.

[scottwilkerson]

Just looking at the system log I an seeing errors for over 60 SNMP traps/sec and the traps look identical

How to do you have these hosts configured to send traps? At this pace you could be receiving thousands a minute, which could really slow down a single system.

In your npcd.log I see this before your restart

Code: Select all

[03-07-2018 11:35:30] NPCD: WARN: MAX load reached: load 683.550000/40.000000 at i=1
[03-07-2018 11:35:45] NPCD: WARN: MAX load reached: load 741.590000/40.000000 at i=1
[03-07-2018 11:36:00] NPCD: WARN: MAX load reached: load 831.490000/40.000000 at i=1
[03-07-2018 11:36:38] NPCD: WARN: MAX load reached: load 745.150000/40.000000 at i=1
[03-07-2018 11:36:53] NPCD: WARN: MAX load reached: load 676.530000/40.000000 at i=1
[03-07-2018 11:37:08] NPCD: WARN: MAX load reached: load 611.300000/40.000000 at i=1
[03-07-2018 11:37:23] NPCD: WARN: MAX load reached: load 550.440000/40.000000 at i=1
[03-07-2018 11:37:54] NPCD: WARN: MAX load reached: load 465.090000/40.000000 at i=1
[03-07-2018 11:38:21] NPCD: WARN: MAX load reached: load 393.590000/40.000000 at i=1
[03-07-2018 11:38:49] NPCD: WARN: MAX load reached: load 341.270000/40.000000 at i=1
[03-07-2018 11:39:04] NPCD: WARN: MAX load reached: load 303.270000/40.000000 at i=1
[03-07-2018 11:39:19] NPCD: WARN: MAX load reached: load 307.120000/40.000000 at i=1
[03-07-2018 11:39:34] NPCD: WARN: MAX load reached: load 333.760000/40.000000 at i=1
[03-07-2018 11:39:49] NPCD: WARN: MAX load reached: load 336.360000/40.000000 at i=1
[03-07-2018 11:40:14] NPCD: WARN: MAX load reached: load 357.650000/40.000000 at i=1

This is why you machine is slow, the load is sometimes reaching 1000. you need to determine what is causing the load, it could be the traps, but likely more than that and hard to decipher because you just restarted 5 minutes before creating the profile.

Being you just restarted

[strmon]

I will look into that snmp issue. Meanwhile attaching fresh profile after weekended run. Have a look and suggest.

[cdienger]

The logs show:

ndo2db: Error: max retries exceeded sending message to queue. Kernel queue parameters may need to be tuned. See README.

Tuning is covered in https://support.nagios.com/kb/article.php?id=139 . Please follow the guide and let us know if you have any further issues.

[scottwilkerson]

also in your /etc/my.cnf you likely want to add

Code: Select all

max-connections=500

under

Code: Select all

[mysqld]

because we are seeing you hit a limit in many php log files.

[strmon]

I have followed the tuning guide and instruction provided on link. But still performance is low. Still getting error for Performance Grapher and Db maintenance ..!

Attaching fresh profile file, have a look.

[scottwilkerson]

you have errors that show your system has run out of memory and shut processes down.

I would start by rebooting the server

[strmon]

PFA , fresh profile file after reboot.