Nagios Support Forum

Posted: **Mon Sep 17, 2018 8:15 am**

Hi,

Our system dashboard is showing as healthy with plenty of available free disk space.

A number of sources are stopping unexpectedly, with the status displaying as stopped.

When trying to start the problematic stopped sources they sometimes start but then stop again.

Have you experienced this behaviour before?

Are there any logs that can be inspected to aid troubleshooting?

thanks

Posted: **Mon Sep 17, 2018 12:48 pm**

Check for any messages regarding nfcapd in /var/log/messages. nfcapd is the process that opens the listening port for the source and when it is running should be visible with:

yum -y install net-tools
netstat -nap | grep nfcapd

Posted: **Mon Oct 01, 2018 4:27 am**

If a number of sources are stopped this allows the previously stopped sources to be started and remain started.

This leads me to suspect that there is max connections issue somewhere.

Can you provide the password for MariaDB root user on Network Analyzer to replace the bold password below:
https://support.nagios.com/kb/article.php?id=513

mysql -uroot -pnagiosxi -e "show variables like 'max_connections';"
mysql -uroot -pnagiosxi -e "show global status like 'Max_used_connections';"

Are there any other settings that may be restricting the maxium number of concurrently started sources?

Thanks

Posted: **Mon Oct 01, 2018 1:52 pm**

The credentials are nagiosna:nagiosna

mysql -unagiosna -pnagiosna -e "show variables like 'max_connections';"
mysql -unagiosna -pnagiosna -e "show global status like 'Max_used_connections';"

How many sources do you have on the machine? I wonder if you may be running into an open file limit - do you see anything logged in /var/log/messages? What is the output of "su - nna --command "ulimit -a""?

Posted: **Thu Oct 04, 2018 2:17 am**

Ok might not be anything to do with max db connections:

Code: Select all

+-----------------+-------+
| Variable_name   | Value |
+-----------------+-------+
| max_connections | 151   |
+-----------------+-------+

Code: Select all

+----------------------+-------+
| Variable_name        | Value |
+----------------------+-------+
| Max_used_connections | 7     |
+----------------------+-------+

Code: Select all

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 255762
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 4096
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Sources Count = 141

fs.file-max = 6505764

Nothing obvious in messages

Posted: **Thu Oct 04, 2018 8:45 am**

One cause of the sources stopping after a few minutes.
It could be a Kernel Semaphore limit that needs to be increased on the NNA server.

To do that edit this file

Code: Select all