Nagios XI Hanging.

[operations_asavie]

Hi,

Could you please help me as a matter of urgency?

I have installed Nagios XI on a newly created VM. Its purpose is to monitor 8 host VMs and all guest VMs that are on each of the 8 hosts. I am using the VMware plugin to do this. Yesterday, I set up monitoring 2 of the 8 hosts and monitoring for all their guest VMs, all was working fine. Today when I try to monitor a third host and all its associated guest VMs, Nagios XI crashes/ hangs. All free memory is used up and all I can do is try to force stop Nagios XI (service nagios stop) and then delete all services for the third host through the GUI and then restart.

Can you please tell me why this is happening?

Any information you need from me, please ask and I will get it over to you straight away.

[rkennedy]

What resources do you have allocated to the machine? Can you post the result of top|head -5?

[operations_asavie]

head|tail -5? Typo I presume?

[rkennedy]

Ah, yes. I'll go get some caffeine. I updated it with the proper command, top|head -5.

[operations_asavie]

See below for the result. Nagios XI has been running for approx 5 mins at this point.

top - 15:31:06 up 6:13, 2 users, load average: 630.08, 268.26, 103.08
Tasks: 760 total, 52 running, 702 sleeping, 0 stopped, 6 zombie
Cpu(s): 26.9%us, 65.9%sy, 0.0%ni, 0.0%id, 5.5%wa, 0.0%hi, 1.7%si, 0.0%st
Mem: 1922252k total, 1867692k used, 54560k free, 592k buffers
Swap: 1675260k total, 1673740k used, 1520k free, 6276k cached

[rkennedy]

Is anything else running on this machine? Can you post the full result of -

Code: Select all

top
lscpu

[operations_asavie]

Output of top command
top - 15:34:29 up 6:17, 2 users, load average: 631.11, 447.38, 203.91
Tasks: 900 total, 15 running, 782 sleeping, 0 stopped, 103 zombie
Cpu(s): 20.5%us, 73.9%sy, 0.0%ni, 0.0%id, 4.3%wa, 0.0%hi, 1.3%si, 0.0%st
Mem: 1922252k total, 1869148k used, 53104k free, 616k buffers
Swap: 1675260k total, 1670372k used, 4888k free, 11396k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
20726 nagios 20 0 10280 596 332 D 12.3 0.0 0:08.27 nagios
20727 nagios 20 0 10276 616 320 D 5.7 0.0 0:14.48 nagios
20728 nagios 20 0 10280 576 332 D 4.4 0.0 0:07.84 nagios
20725 nagios 20 0 10276 660 356 R 1.6 0.0 0:06.95 nagios
21573 nagios 20 0 126m 4304 452 D 0.7 0.2 0:00.11 check_esx3.pl
21651 nagios 20 0 129m 5564 544 D 0.7 0.3 0:00.10 check_esx3.pl
21659 nagios 20 0 126m 4504 444 D 0.7 0.2 0:00.10 check_esx3.pl
21660 nagios 20 0 126m 4808 444 D 0.7 0.3 0:00.10 check_esx3.pl
21666 nagios 20 0 126m 5656 440 D 0.7 0.3 0:00.10 check_esx3.pl
21680 nagios 20 0 129m 5584 480 D 0.7 0.3 0:00.10 check_esx3.pl
21682 nagios 20 0 126m 5288 488 D 0.7 0.3 0:00.10 check_esx3.pl
2510 mysql 20 0 722m 12m 1832 S 0.6 0.7 2:15.08 mysqld
20730 nagios 20 0 50096 616 388 S 0.6 0.0 0:01.62 ndo2db
21569 nagios 20 0 125m 3928 588 D 0.6 0.2 0:00.09 check_esx3.pl
21579 nagios 20 0 127m 5504 528 D 0.6 0.3 0:00.10 check_esx3.pl
21581 nagios 20 0 126m 5144 464 D 0.6 0.3 0:00.09 check_esx3.pl
21595 nagios 20 0 125m 5040 460 D 0.6 0.3 0:00.09 check_esx3.pl
21600 nagios 20 0 129m 6072 440 D 0.6 0.3 0:00.09 check_esx3.pl
21622 nagios 20 0 129m 5692 436 D 0.6 0.3 0:00.09 check_esx3.pl
21624 nagios 20 0 126m 5732 436 D 0.6 0.3 0:00.09 check_esx3.pl
21629 nagios 20 0 125m 5556 524 R 0.6 0.3 0:00.09 check_esx3.pl
21631 nagios 20 0 125m 4880 416 R 0.6 0.3 0:00.09 check_esx3.pl
21658 nagios 20 0 126m 4824 460 D 0.6 0.3 0:00.09 check_esx3.pl
21671 nagios 20 0 129m 4828 496 D 0.6 0.3 0:00.09 check_esx3.pl
21675 nagios 20 0 126m 4048 408 D 0.6 0.2 0:00.09 check_esx3.pl
21676 nagios 20 0 125m 5508 1108 D 0.6 0.3 0:00.09 check_esx3.pl
21677 nagios 20 0 126m 5556 416 D 0.6 0.3 0:00.09 check_esx3.pl
21684 nagios 20 0 126m 5788 588 R 0.6 0.3 0:00.09 check_esx3.pl
21686 nagios 20 0 129m 5568 448 D 0.6 0.3 0:00.09 check_esx3.pl
21692 nagios 20 0 127m 6064 464 D 0.6 0.3 0:00.09 check_esx3.pl
21722 nagios 20 0 125m 5476 420 D 0.6 0.3 0:00.09 check_esx3.pl
21736 nagios 20 0 129m 6060 468 D 0.6 0.3 0:00.09 check_esx3.pl
21737 nagios 20 0 125m 5836 1044 R 0.6 0.3 0:00.09 check_esx3.pl
21739 nagios 20 0 126m 5424 448 D 0.6 0.3 0:00.09 check_esx3.pl
21740 nagios 20 0 124m 3784 404 D 0.6 0.2 0:00.09 check_esx3.pl
21303 nagios 20 0 308m 13m 5320 D 0.6 0.7 0:00.11 php
21321 nagios 20 0 309m 14m 5312 D 0.6 0.8 0:00.11 php
21426 nagios 20 0 306m 11m 5284 D 0.6 0.6 0:00.10 php
21427 nagios 20 0 309m 14m 5312 D 0.6 0.8 0:00.09 php
21468 nagios 20 0 306m 12m 5280 D 0.6 0.7 0:00.09 php
21599 nagios 20 0 125m 4108 404 D 0.6 0.2 0:00.08 check_esx3.pl
21602 nagios 20 0 125m 3936 404 D 0.6 0.2 0:00.08 check_esx3.pl
21611 nagios 20 0 127m 6284 476 D 0.6 0.3 0:00.08 check_esx3.pl

Output of lscpu command.

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 1
On-line CPU(s) list: 0
Thread(s) per core: 1
Core(s) per socket: 1
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 45
Stepping: 7
CPU MHz: 1999.999
BogoMIPS: 3999.99
Hypervisor vendor: VMware
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 20480K
NUMA node0 CPU(s): 0

[hsmith]

Perl checks can be very taxing on your server. I see that you're only running with 1 2GHz CPU. I would recommend that you add at least 3 more CPUs to this machine to try to help balance this load on it a little bit better. Adding at least another 2GB of ram will be ideal as well. Sure, you don't have a lot of checks, but the ones that you do use A LOT of resources.

[hsmith]

I just noticed that you have a ticket open for the same issue:

A couple of things here:

This was posted in general support. We have a forum for customers, if you don't have access to it, please email [email protected]
We like to keep troubleshooting consolidated to one place, that way our technicians are not repeating their work, and having you send us things that we have already asked for.

I am going to lock this thread since a ticket was sent in. Please resume all communication there.