Page 1 of 1

NSCA Daemon performance problems

Posted: Mon Apr 07, 2014 4:25 pm
by sujitt
When we call send_nsca in batch may be 100s in less than a second. Lot of them do not show up on nagios. Is there a way to make the nsca daemon more performant or queue the requests faster and not fail with error connection closed before receiving packet.

here is our setup

agios XI Version : 2012R2.5
gaatlp81u 2.6.32-358.2.1.el6.x86_64 x86_64
CentOS release 6.4 (Final)
Gnome is not installed
Apache Information

PHP Version: 5.3.3
Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.154 Safari/537.36
Server Name: gaatlp81u
Server Address: 10.2.2.100
Server Port: 80
Date/Time

PHP Timezone: America/New_York
PHP Time: Mon, 07 Apr 2014 17:24:19 -0400
System Time: Mon, 07 Apr 2014 17:24:19 -0400
Nagios XI Data

nagios (pid 17051) is running...
NPCD running (pid 1570).
ndo2db (pid 1626) is running...
CPU Load 15: 0.69
Total Hosts: 16
Total Services: 2155
Function 'get_base_uri' returns: http://gaatlp81u/nagiosxi/
Function 'get_base_url' returns: http://gaatlp81u/nagiosxi/
Function 'get_backend_url(internal_call=false)' returns: http://gaatlp81u/nagiosxi/includes/comp ... rofile.php
Function 'get_backend_url(internal_call=true)' returns: http://localhost/nagiosxi/backend/
Ping Test localhost

Running:
/bin/ping -c 3 localhost 2>&1
PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.022 ms
64 bytes from localhost (127.0.0.1): icmp_seq=2 ttl=64 time=0.030 ms
64 bytes from localhost (127.0.0.1): icmp_seq=3 ttl=64 time=0.027 ms

--- localhost ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.022/0.026/0.030/0.005 ms
Test wget To locahost

WGET From URL: http://localhost/nagiosql/index.php
Running:
/usr/bin/wget http://localhost/nagiosql/index.php
--2014-04-07 17:24:21-- http://localhost/nagiosql/index.php
Resolving localhost... ::1, 127.0.0.1
Connecting to localhost|::1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5259 (5.1K) [text/html]
Saving to: `/tmp/nagiosql_index.tmp'

0K ..... 100% 436M=0s

2014-04-07 17:24:21 (436 MB/s) - `/tmp/nagiosql_index.tmp' saved [5259/5259]

Re: NSCA Daemon performance problems

Posted: Mon Apr 07, 2014 6:17 pm
by scottwilkerson
What you will need to do is edit the /etc/xinetd.d/nsca file on the XI server to include the following lines

Code: Select all

per_source = UNLIMITED
instances = UNLIMITED
add them between { and }

Then restart xinetd

Code: Select all

service xinetd restart

Re: NSCA Daemon performance problems

Posted: Tue Apr 08, 2014 1:54 pm
by sujitt
I am trying to test these settings in a test nagiosxi server and I am receiving an error
0 data packets received
This was testing send_nsca from local server
bot send_nsca and nsca are 2.9.1 version

Re: NSCA Daemon performance problems

Posted: Tue Apr 08, 2014 3:51 pm
by slansing
Well, Nagios does not work by scheduling all tasks at once, it sprinkles them out over a period of time so they do not but up against each other. Do you have success with calling half that number, or even a quarter? The limits Scott posted should help out on the NSCA side of things, but this may not work as you expect it. I did notice you hinted that (some) may get through.

Re: NSCA Daemon performance problems

Posted: Tue Apr 08, 2014 3:56 pm
by sujitt
I am trying to test this in the test environment and having challenges with setting up NSCA. This is new nagios xi server and I thought it comes installed with NSCA and all firewall rules for it.
I see that the iptables list does not show nsca or nrpe there. help me configure this

Thanks

Re: NSCA Daemon performance problems

Posted: Wed Apr 09, 2014 10:54 am
by lmiltchev
Open the "iptables-config" in a text editor:

Code: Select all

vi /etc/sysconfig/iptables-config
and make sure you have:

Code: Select all

IPTABLES_SAVE_ON_STOP="yes"
and

Code: Select all

IPTABLES_SAVE_ON_RESTART="yes"
Save and exit. Stop iptables:

Code: Select all

service iptables stop
Open the "iptables" in a text editor:

Code: Select all

vi /etc/sysconfig/iptables
and add the following lines:

Code: Select all

-A INPUT -m state --state NEW -m tcp -p tcp --dport 5666 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 5667 -j ACCEPT
Save and exit. Start the iptables:

Code: Select all

service iptables start
Check if the rules are added:

Code: Select all

iptables -L -n