Page 1 of 1

Adding switch fails after about 600 services?

Posted: Fri Mar 20, 2015 7:47 am
by GhostRider2110
Having a strange problem. We were adding a network switch. We started using the Wizard to add the switch, and it would hum along in the verification for a good 2min then come back saying it failed. No real reason given. So we decided to use the Wizard to add things in batches, figuring we were just running into a timeout on the scripts(s). Things work fine up until around the 600 mark on number of services for that host (being a switch). Then the verification fails and the host does not even show up. Delete those just added, and the host comes back, and all seems to be fine.

Thanks
Mitch


Nagios XI Installation Profile

Download Profile
System:

Nagios XI Version : 2014R2.6
iganagios.iga.local 2.6.32-504.3.3.el6.x86_64 x86_64
CentOS release 6.6 (Final)
Gnome is not installed
Apache Information

PHP Version: 5.3.3
Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36
Server Name: iganagios.iga.local
Server Address: 10.100.30.8
Server Port: 80
Date/Time

PHP Timezone: America/New_York
PHP Time: Fri, 20 Mar 2015 08:46:06 -0400
System Time: Fri, 20 Mar 2015 08:46:06 -0400
Nagios XI Data

License ends in: QRQSQP

nagios (pid 25103) is running...
NPCD running (pid 2619).
ndo2db (pid 1691) is running...
CPU Load 15: 2.13
Total Hosts: 154
Total Services: 5730
Function 'get_base_uri' returns: http://iganagios.iga.local/nagiosxi/
Function 'get_base_url' returns: http://iganagios.iga.local/nagiosxi/
Function 'get_backend_url(internal_call=false)' returns: http://iganagios.iga.local/nagiosxi/inc ... rofile.php
Function 'get_backend_url(internal_call=true)' returns: http://localhost/nagiosxi/backend/
Ping Test localhost

Running:
/bin/ping -c 3 localhost 2>&1
PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.044 ms
64 bytes from localhost (127.0.0.1): icmp_seq=2 ttl=64 time=0.023 ms
64 bytes from localhost (127.0.0.1): icmp_seq=3 ttl=64 time=0.050 ms

--- localhost ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 0.023/0.039/0.050/0.011 ms
Test wget To localhost

WGET From URL: http://localhost/nagiosxi/includes/components/ccm/
Running:
/usr/bin/wget http://localhost/nagiosxi/includes/components/ccm/
--2015-03-20 08:46:08-- http://localhost/nagiosxi/includes/components/ccm/
Resolving localhost... ::1, 127.0.0.1
Connecting to localhost|::1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: "/usr/local/nagiosxi/tmp/ccm_index.tmp"

0K ........ 360M=0s

2015-03-20 08:46:08 (360 MB/s) - "/usr/local/nagiosxi/tmp/ccm_index.tmp" saved [8385]

Network Settings

1: lo: mtu 65536 qdisc noqueue state UNKNOWN

link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

inet 127.0.0.1/8 scope host lo

inet6 ::1/128 scope host

valid_lft forever preferred_lft forever

2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000

link/ether 00:50:56:a3:63:66 brd ff:ff:ff:ff:ff:ff

inet 10.100.30.8/24 brd 10.100.30.255 scope global eth0

inet6 fe80::250:56ff:fea3:6366/64 scope link

valid_lft forever preferred_lft forever


Kernel IP routing table

Destination Gateway Genmask Flags Metric Ref Use Iface

10.100.30.0 * 255.255.255.0 U 0 0 0 eth0

link-local * 255.255.0.0 U 1002 0 0 eth0

default 10.100.30.1 0.0.0.0 UG 0 0 0 eth0

Re: Adding switch fails after about 600 services?

Posted: Fri Mar 20, 2015 9:25 am
by tmcdonald
Please run the following and post the results:

Code: Select all

grep "memory_limit" /etc/php.ini
grep "max_execution_time" /etc/php.ini
grep "max_input_time" /etc/php.ini

Re: Adding switch fails after about 600 services?

Posted: Fri Mar 20, 2015 11:16 am
by GhostRider2110

Code: Select all

# grep "memory_limit" /etc/php.ini
memory_limit = 128M
# grep "max_execution_time" /etc/php.ini
max_execution_time = 30     
# grep "max_input_time" /etc/php.ini
; max_input_time
max_input_time = 60
More info on the device:

That switch is a Cisco Nexus 5596 with 5 FEX’s attached. Each FEX has 48 copper ports and 4 10GB uplinks back to the 5596. So a very large number of ports.

Thanks
Mitch

Re: Adding switch fails after about 600 services?

Posted: Fri Mar 20, 2015 1:47 pm
by lmiltchev
Try increasing all of these values (memory_limit, max_execution_time, and max_input_time). Also, add the following line on the bottom of the "/etc/php.ini" file:

Code: Select all

max_input_vars = 100000
Restart apache:

Code: Select all

service httpd restart
Let us know if this fixed your issue.

Re: Adding switch fails after about 600 services?

Posted: Sun Mar 22, 2015 10:20 pm
by Box293
The value in /etc/php.ini that needs changing is:

Code: Select all

max_execution_time
Try setting it to:

Code: Select all

max_execution_time = 600 
Then:

Code: Select all

service httpd restart

Re: Adding switch fails after about 600 services?

Posted: Tue Mar 24, 2015 7:22 am
by GhostRider2110
Thanks, make suggested changes and will be testing. --Mitch

Re: Adding switch fails after about 600 services?

Posted: Tue Mar 24, 2015 9:25 am
by lmiltchev
We will keep this thread open for a while in case you have more questions/issues.

Re: Adding switch fails after about 600 services?

Posted: Tue Mar 24, 2015 11:01 am
by GhostRider2110
Changes made and all is working fine. Thanks! --Mitch