Page 2 of 2
Re: Nagios Load very high
Posted: Mon Feb 24, 2020 2:05 pm
by jdunitz
According to your output, you have the default number of connections (100) and you're using them all plus one (101). Increasing your connection limit to 1000 as previously suggested should help with your problem.
See:
Nagios XI - MySQL/MariaDB - Max Connections
Let us know if this does it for you.
--Jeffrey
Re: Nagios Load very high
Posted: Wed Mar 04, 2020 7:49 am
by lgaddam
Hi Jeff,
To increase max connections , please provide me steps. Im not able to get actual steps from the provided link.
and also let me know whether i have to do repairdatabases.sh script after doing modification or reboot is required or any services need to be started to get affect.
Re: Nagios Load very high
Posted: Wed Mar 04, 2020 10:48 am
by jdunitz
Was your Max_used_connections output from before or after you changed the connection limit and rebooted?
If it was after, it doesn't seem to have taken effect for some reason, and we'd want to figure out what went wrong there.
If it was before, please apply the changes and reboot and see how it's doing.
Re: Nagios Load very high
Posted: Wed Mar 04, 2020 11:02 am
by lmiltchev
To increase max connections , please provide me steps. Im not able to get actual steps from the provided link.
Have you modified the
my.cnf file by adding the following lines in the section [mysqld]?
Code: Select all
max_connections=1000
open_files_limit = 4096
Can you show us what you have in that file?
and also let me know whether i have to do repairdatabases.sh script after doing modification or reboot is required or any services need to be started to get affect.
Running the repair on the database won't hurt anything, but you don't have to do it, unless you have crashed tables/errors in the mysqld.log (or mariadb.log).
Re: Nagios Load very high
Posted: Wed Mar 11, 2020 2:45 am
by lgaddam
Thanks I have done the changes in /etc/my.cnf & restarted mysqld as per procedure provided by below link.
https://support.nagios.com/kb/article.php?id=513
Now its showing the max connections as 1000.
[root@nagiosp01 httpd]# mysql -uroot -pnagiosxi -e "show variables like 'max_connections';"
+-----------------+-------+
| Variable_name | Value |
+-----------------+-------+
| max_connections | 1000 |
+-----------------+-------+
You have new mail in /var/spool/mail/root
[root@glnagiosp01 httpd]#
Re: Nagios Load very high
Posted: Wed Mar 11, 2020 9:21 am
by lmiltchev
Great! You can reboot now, and see if your server will perform better now.
If the high load is back, PM us your latest profile, so that we can review your logs. Thanks!
Re: Nagios Load very high
Posted: Thu Mar 12, 2020 9:22 am
by lgaddam
Hi,
Sorry this didn't helped, we identified the issue three times after doing this change, high load & high process count observed.
We observed this issue today when we do apply config as well as when we didn't do apply config & also observed monitoring engine flapping sometimes.
But main issue is that, when load is high, bulk of node down alerts are generating, not sure why these false alerts are generating, its creating panic to IT teams. We have to fix this ASAP.
After running dbrepair.sh script the load came down within 20 minutes. Don't know is it the right way to do to decrease load but I am doing this everytime we find high load, not sure everytime if we run this script whether any other complications will be araised ?
And also I observed that when we run this script, Nagios process more taking time to stop.
But we haven't rebooted the server, this is physical server , so Linux team not recommending as of now. Is really required a reboot ?
If require reboot then we need to provide proper justification because same hosts(3646) & services(14780) are also present in secondary and did not find any load issues on that.
Please find the screenshot of the load graph, process count graph and also profile.zip attached.
Moderator's Note: The profile has been shared with the support team but has been removed from the public forum.
Re: Nagios Load very high
Posted: Thu Mar 12, 2020 11:51 am
by lmiltchev
A remote session may be needed in order to further troubleshoot this issue. Please open a new support ticket via our support center here:
https://support.nagios.com/tickets/