Adding switch fails after about 600 services?

[GhostRider2110]

Having a strange problem. We were adding a network switch. We started using the Wizard to add the switch, and it would hum along in the verification for a good 2min then come back saying it failed. No real reason given. So we decided to use the Wizard to add things in batches, figuring we were just running into a timeout on the scripts(s). Things work fine up until around the 600 mark on number of services for that host (being a switch). Then the verification fails and the host does not even show up. Delete those just added, and the host comes back, and all seems to be fine.

Thanks
Mitch


Nagios XI Installation Profile

Download Profile
System:

Nagios XI Version : 2014R2.6
iganagios.iga.local 2.6.32-504.3.3.el6.x86_64 x86_64
CentOS release 6.6 (Final)
Gnome is not installed
Apache Information

PHP Version: 5.3.3
Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36
Server Name: iganagios.iga.local
Server Address: 10.100.30.8
Server Port: 80
Date/Time

PHP Timezone: America/New_York
PHP Time: Fri, 20 Mar 2015 08:46:06 -0400
System Time: Fri, 20 Mar 2015 08:46:06 -0400
Nagios XI Data

License ends in: QRQSQP

nagios (pid 25103) is running...
NPCD running (pid 2619).
ndo2db (pid 1691) is running...
CPU Load 15: 2.13
Total Hosts: 154
Total Services: 5730
Function 'get_base_uri' returns: http://iganagios.iga.local/nagiosxi/
Function 'get_base_url' returns: http://iganagios.iga.local/nagiosxi/
Function 'get_backend_url(internal_call=false)' returns: http://iganagios.iga.local/nagiosxi/inc ... rofile.php
Function 'get_backend_url(internal_call=true)' returns: http://localhost/nagiosxi/backend/
Ping Test localhost

Running:
/bin/ping -c 3 localhost 2>&1
PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.044 ms
64 bytes from localhost (127.0.0.1): icmp_seq=2 ttl=64 time=0.023 ms
64 bytes from localhost (127.0.0.1): icmp_seq=3 ttl=64 time=0.050 ms

--- localhost ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 0.023/0.039/0.050/0.011 ms
Test wget To localhost

WGET From URL: http://localhost/nagiosxi/includes/components/ccm/
Running:
/usr/bin/wget http://localhost/nagiosxi/includes/components/ccm/
--2015-03-20 08:46:08-- http://localhost/nagiosxi/includes/components/ccm/
Resolving localhost... ::1, 127.0.0.1
Connecting to localhost|::1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: "/usr/local/nagiosxi/tmp/ccm_index.tmp"

0K ........ 360M=0s

2015-03-20 08:46:08 (360 MB/s) - "/usr/local/nagiosxi/tmp/ccm_index.tmp" saved [8385]

Network Settings

1: lo: mtu 65536 qdisc noqueue state UNKNOWN

link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

inet 127.0.0.1/8 scope host lo

inet6 ::1/128 scope host

valid_lft forever preferred_lft forever

2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000

link/ether 00:50:56:a3:63:66 brd ff:ff:ff:ff:ff:ff

inet 10.100.30.8/24 brd 10.100.30.255 scope global eth0

inet6 fe80::250:56ff:fea3:6366/64 scope link

valid_lft forever preferred_lft forever


Kernel IP routing table

Destination Gateway Genmask Flags Metric Ref Use Iface

10.100.30.0 * 255.255.255.0 U 0 0 0 eth0

link-local * 255.255.0.0 U 1002 0 0 eth0

default 10.100.30.1 0.0.0.0 UG 0 0 0 eth0

[tmcdonald]

Please run the following and post the results:

Code: Select all

grep "memory_limit" /etc/php.ini
grep "max_execution_time" /etc/php.ini
grep "max_input_time" /etc/php.ini

[GhostRider2110]

Code: Select all

# grep "memory_limit" /etc/php.ini
memory_limit = 128M
# grep "max_execution_time" /etc/php.ini
max_execution_time = 30     
# grep "max_input_time" /etc/php.ini
; max_input_time
max_input_time = 60

More info on the device:

That switch is a Cisco Nexus 5596 with 5 FEX’s attached. Each FEX has 48 copper ports and 4 10GB uplinks back to the 5596. So a very large number of ports.

Thanks
Mitch

[lmiltchev]

Try increasing all of these values (memory_limit, max_execution_time, and max_input_time). Also, add the following line on the bottom of the "/etc/php.ini" file:

Code: Select all

max_input_vars = 100000

Restart apache:

Code: Select all

service httpd restart

Let us know if this fixed your issue.

[Box293]

The value in /etc/php.ini that needs changing is:

Code: Select all

max_execution_time

Try setting it to:

Code: Select all

max_execution_time = 600 

Then:

Code: Select all

service httpd restart

[GhostRider2110]

Thanks, make suggested changes and will be testing. --Mitch

[lmiltchev]

We will keep this thread open for a while in case you have more questions/issues.

[GhostRider2110]

Changes made and all is working fine. Thanks! --Mitch