Nagios XI Performance Issues-RAM Utilization

This support forum board is for support questions relating to Nagios XI, our flagship commercial network monitoring solution.
anoop
Posts: 95
Joined: Tue Jun 25, 2013 1:22 am

Nagios XI Performance Issues-RAM Utilization

Post by anoop »

Dear Team,

We have installed Nagios XI Server in CentOS 6.3 64 bit with 32GB of RAM and 16 core processor.

Now, we have added 350 hosts with total 3100 Services which are all active checks but my RAM Utilization in going upto 14GB maximum for 3100 services and CPU Utilization is not crossing maximum of 7%, Then we rebooted our Nagios XI Server and RAM Utiilization went down to 3GB .. and even cache memory is utlizing 6GB ..


Please let us know the behaviour of my server and suggest us better performance..


Thanks in advance
System:
Nagios XI Version : 2012R2.2 | PHP Version: 5.3.3
Offloaded MySQL DB on another virtual machine
16 CPU with 2 cores each | 32 GB RAM | 1 TB HDD
CentOS-6.3 |Total = 4,000 hosts| 40,000 services.
slansing
Posts: 7698
Joined: Mon Apr 23, 2012 4:28 pm
Location: Travelling through time and space...

Re: Nagios XI Performance Issues-RAM Utilization

Post by slansing »

This is a bit odd, what types of checks are you running? How many are active or passive, and what plugins are they using? i.e. active SNMP checks? Nrpe checks? Etc.. Do you have anything else running on the system besides nagios XI? Have you taken a look at your "TOP" and tried to find where the memory is being held or allocated to? Are you sure this is not just SWAP?
anoop
Posts: 95
Joined: Tue Jun 25, 2013 1:22 am

Re: Nagios XI Performance Issues-RAM Utilization

Post by anoop »

hi,

Active Service Checks
1-min 607
5-min 3116
15-min 3247

No passive checks

we auto discovered the 338 machines with default services and only one one switch with SNMP monitor & two machines with NRPE and one machine with NRDS.
Only Nagios Xi server installed and we offloaded Database to other machine. For top command only apache and nagios exists..
System:
Nagios XI Version : 2012R2.2 | PHP Version: 5.3.3
Offloaded MySQL DB on another virtual machine
16 CPU with 2 cores each | 32 GB RAM | 1 TB HDD
CentOS-6.3 |Total = 4,000 hosts| 40,000 services.
slansing
Posts: 7698
Joined: Mon Apr 23, 2012 4:28 pm
Location: Travelling through time and space...

Re: Nagios XI Performance Issues-RAM Utilization

Post by slansing »

How about the other question, have you been able to track where that memory is being allocated? And are you sure it is not dedicated to swap? Is it possible you still have an autodiscovery job running in the background?
anoop
Posts: 95
Joined: Tue Jun 25, 2013 1:22 am

Re: Nagios XI Performance Issues-RAM Utilization

Post by anoop »

Hi,

We have given swap memory as 2GB, it is not dedicated to swap... and there are no auto discovery jobs are running.. but we given storage as 50GB and does it creating any problem with the storage..??
System:
Nagios XI Version : 2012R2.2 | PHP Version: 5.3.3
Offloaded MySQL DB on another virtual machine
16 CPU with 2 cores each | 32 GB RAM | 1 TB HDD
CentOS-6.3 |Total = 4,000 hosts| 40,000 services.
sreinhardt
-fno-stack-protector
Posts: 4366
Joined: Mon Nov 19, 2012 12:10 pm

Re: Nagios XI Performance Issues-RAM Utilization

Post by sreinhardt »

So this memory issue was not happening after you started the autodiscovery task that you mentioned in your other thread? 50GB would not be an issue, however if you have 32GB ram, it is normally suggested to have around that in swap space as well. It normally wouldn't be needed just by nagios, unless you have a very large install. Also, do you happen to be scheduling any reports when this memory issue happens?
Nagios-Plugins maintainer exclusively, unless you have other C language bugs with open-source nagios projects, then I am happy to help! Please pm or use other communication to alert me to issues as I no longer track the forum.
anoop
Posts: 95
Joined: Tue Jun 25, 2013 1:22 am

Re: Nagios XI Performance Issues-RAM Utilization

Post by anoop »

Hi Team,,

Im attaching my status files and please explain us where we are doing mistake...

We didn't given any scheduling reports and if i reboot my server, RAM utilization in going down drastically..

Suggest us how to reduce the RAM Utilization for a better performance..

Thanks
You do not have the required permissions to view the files attached to this post.
System:
Nagios XI Version : 2012R2.2 | PHP Version: 5.3.3
Offloaded MySQL DB on another virtual machine
16 CPU with 2 cores each | 32 GB RAM | 1 TB HDD
CentOS-6.3 |Total = 4,000 hosts| 40,000 services.
sreinhardt
-fno-stack-protector
Posts: 4366
Joined: Mon Nov 19, 2012 12:10 pm

Re: Nagios XI Performance Issues-RAM Utilization

Post by sreinhardt »

At this point, it has only been stated that memory is being used heavily, not what is using the memory. Could you post the output of the following commands please.

Code: Select all

top 1
ps -euf
ps -amcwwwxo "command %mem %cpu" | grep -v grep | head -25
Nagios-Plugins maintainer exclusively, unless you have other C language bugs with open-source nagios projects, then I am happy to help! Please pm or use other communication to alert me to issues as I no longer track the forum.
anoop
Posts: 95
Joined: Tue Jun 25, 2013 1:22 am

Re: Nagios XI Performance Issues-RAM Utilization

Post by anoop »

HI Team,

As per ur suggestion i have taken the outputs of

ps -euf

top 1--------command not working...... i have taken output for "top"
ps -euf------ attached output ... please find the attachment.
ps -amcwwwxo "command %mem %cpu" | grep -v grep | head -25----------- command is not working... please let me know the command exactly..


Now my RAM Utilization went to 22GB out of 32GB..

Not able to understand from where the utilization is taking..

Please help us out is solving the issue..

Thanks
You do not have the required permissions to view the files attached to this post.
System:
Nagios XI Version : 2012R2.2 | PHP Version: 5.3.3
Offloaded MySQL DB on another virtual machine
16 CPU with 2 cores each | 32 GB RAM | 1 TB HDD
CentOS-6.3 |Total = 4,000 hosts| 40,000 services.
abrist
Red Shirt
Posts: 8334
Joined: Thu Nov 15, 2012 1:20 pm

Re: Nagios XI Performance Issues-RAM Utilization

Post by abrist »

Both of those screenshots show very little memory usage. Lets check how much of those 10gbs are disk caching:

Code: Select all

free -m
Former Nagios employee
"It is turtles. All. The. Way. Down. . . .and maybe an elephant or two."
VI VI VI - The editor of the Beast!
Come to the Dark Side.
Locked