Page 1 of 1
Nagios running slowly and utilizing 100% RAM
Posted: Fri May 21, 2021 8:01 am
by taashee
Hello Team ,
It is observed that Nagios has become too slow . On checking it is found that it is using almost whole RAM . We took reboot of the server also . We even tried killall -u nagios .
And started back the service .It frees the RAM , but again RAM utilization came to 100 % . We have also configured modgearman and offloaded the DB also to reduce the load .
For your reference please find the profile .
Apart from this , it indicates the message too often that license has expired and we have to go to license settings and mark the key again . What could be the possible reason for this . This has happened twice .
Moderator's Note: The profile has been shared with the support team but has been removed from the public forum.
Re: Nagios running slowly and utilizing 100% RAM
Posted: Fri May 21, 2021 2:29 pm
by dchurch
Looks like you're running an older version of Nagios XI (5.5.7).
Your system appears to be running into a known issue where the event handler is being concurrently run when it shouldn't be. Only one copy of the event handler should be running at once, and it uses a lock file to prevent multiple copies running concurrently, but there's a bug. The result is more slowness, which compounds because the event handler takes event longer to run because the system is bogged down, leading to ~6 copies running on your system at once.
Upgrading Nagios XI may fix this problem. This issue was fixed in
Nagios XI 5.6.1:
5.6.1 - 04/30/2019
- Fixed issue with event_handler.php where the lock file would not be overwritten and stopped notifications being sent [TPS#14180] -JO
There have been
numerous other security patches and performance improvements since 5.5.7 too, and the upgrade is free, so it's probably worth your time to look into upgrading Nagios XI.
Increasing performance
On long-running systems with mucho checks, the database can get bogged down with excessive "paper trail" type data and the software's database queries aren't properly utilizing indexes. It just needs better thresholds to get performance back where it should be:
Open Admin => Performance Settings, then click on the Databases tab. Change the following settings:
- Max Log Entries Age: change to 10
- Max Audit Log Age: change to 10
- Max State History Age: change to 30
It might take up to a day for the "cleaner" process to run depending on how your system is configured, but it'll eventually run and clean your database of all these for you.
See this document:
Nagios XI Database Optimization
Re: Nagios running slowly and utilizing 100% RAM
Posted: Wed May 26, 2021 9:03 am
by taashee
Hello Team ,
Whatever changes you have suggested we have made the changes . Please suggest the best way to update it . Currently we are running 5.5.8 version and db is also offloaded . We are not running modgearman .Please suggest the way we have to update it . System profile already shared .
Re: Nagios running slowly and utilizing 100% RAM
Posted: Wed May 26, 2021 10:53 am
by dchurch
taashee wrote:Currently we are running 5.5.8 version and db is also offloaded .
That isn't what I'm seeing; the database is hosted on the same machine as the Nagios XI monitoring engine. I'm also seeing that you're running Nagios XI 5.7.2, not 5.5.8.
What is the output from the following command?
Code: Select all
mysql -uroot -pnagiosxi -e "show full processlist;"
Re: Nagios running slowly and utilizing 100% RAM
Posted: Wed May 26, 2021 11:09 am
by taashee
Hello team ,
The database you are seeing is older database which is as it is untouched after doing the db offloading . The database was offloaded later on it's not from starting .
Re: Nagios running slowly and utilizing 100% RAM
Posted: Wed May 26, 2021 12:43 pm
by dchurch
Your profile states "localhost" for all three of the mysql databases, nagiosxi, ndoutils, and nagiosql:
Code: Select all
$cfg['db_info']=array(
"nagiosxi" => array(
"dbtype" => 'mysql',
"dbserver" => 'localhost',
"user" => 'nagiosxi',
"db" => 'nagiosxi',
"ndoutils" => array(
"dbtype" => 'mysql',
"dbserver" => 'localhost',
"user" => 'ndoutils',
"db" => 'nagios',
"nagiosql" => array(
"dbtype" => 'mysql',
"dbserver" => 'localhost',
"user" => 'nagiosql',
"db" => 'nagiosql',
);
Perhaps the profile you pulled was from a different server from the Nagios XI install you're having problems with. Did you install Nagios XI on two different servers, and are you using one as the offloaded database host?
The profile also shows MySQL using CPU cycles, so there's activity on it. Whether that's from internal or external connections, something is using that database.