Nagios running slowly and utilizing 100% RAM

This support forum board is for support questions relating to Nagios XI, our flagship commercial network monitoring solution.
Locked
taashee
Posts: 17
Joined: Mon Apr 20, 2020 6:02 am

Nagios running slowly and utilizing 100% RAM

Post by taashee »

Hello Team ,

It is observed that Nagios has become too slow . On checking it is found that it is using almost whole RAM . We took reboot of the server also . We even tried killall -u nagios .

And started back the service .It frees the RAM , but again RAM utilization came to 100 % . We have also configured modgearman and offloaded the DB also to reduce the load .

For your reference please find the profile .

Apart from this , it indicates the message too often that license has expired and we have to go to license settings and mark the key again . What could be the possible reason for this . This has happened twice .

Moderator's Note: The profile has been shared with the support team but has been removed from the public forum.
dchurch
Posts: 858
Joined: Wed Oct 07, 2020 12:46 pm
Location: Yo mama

Re: Nagios running slowly and utilizing 100% RAM

Post by dchurch »

Looks like you're running an older version of Nagios XI (5.5.7).

Your system appears to be running into a known issue where the event handler is being concurrently run when it shouldn't be. Only one copy of the event handler should be running at once, and it uses a lock file to prevent multiple copies running concurrently, but there's a bug. The result is more slowness, which compounds because the event handler takes event longer to run because the system is bogged down, leading to ~6 copies running on your system at once.

Upgrading Nagios XI may fix this problem. This issue was fixed in Nagios XI 5.6.1:
5.6.1 - 04/30/2019
  • Fixed issue with event_handler.php where the lock file would not be overwritten and stopped notifications being sent [TPS#14180] -JO
There have been numerous other security patches and performance improvements since 5.5.7 too, and the upgrade is free, so it's probably worth your time to look into upgrading Nagios XI.

Increasing performance

On long-running systems with mucho checks, the database can get bogged down with excessive "paper trail" type data and the software's database queries aren't properly utilizing indexes. It just needs better thresholds to get performance back where it should be:

Open Admin => Performance Settings, then click on the Databases tab. Change the following settings:

- Max Log Entries Age: change to 10
- Max Audit Log Age: change to 10
- Max State History Age: change to 30

It might take up to a day for the "cleaner" process to run depending on how your system is configured, but it'll eventually run and clean your database of all these for you.

See this document: Nagios XI Database Optimization
If you didn't get an 8% raise over the course of the pandemic, you took a pay cut.

Discussion of wages is protected speech under the National Labor Relations Act, and no employer can tell you you can't disclose your pay with your fellow employees.
taashee
Posts: 17
Joined: Mon Apr 20, 2020 6:02 am

Re: Nagios running slowly and utilizing 100% RAM

Post by taashee »

Hello Team ,

Whatever changes you have suggested we have made the changes . Please suggest the best way to update it . Currently we are running 5.5.8 version and db is also offloaded . We are not running modgearman .Please suggest the way we have to update it . System profile already shared .
dchurch
Posts: 858
Joined: Wed Oct 07, 2020 12:46 pm
Location: Yo mama

Re: Nagios running slowly and utilizing 100% RAM

Post by dchurch »

taashee wrote:Currently we are running 5.5.8 version and db is also offloaded .
That isn't what I'm seeing; the database is hosted on the same machine as the Nagios XI monitoring engine. I'm also seeing that you're running Nagios XI 5.7.2, not 5.5.8.

What is the output from the following command?

Code: Select all

mysql -uroot -pnagiosxi -e "show full processlist;"
If you didn't get an 8% raise over the course of the pandemic, you took a pay cut.

Discussion of wages is protected speech under the National Labor Relations Act, and no employer can tell you you can't disclose your pay with your fellow employees.
taashee
Posts: 17
Joined: Mon Apr 20, 2020 6:02 am

Re: Nagios running slowly and utilizing 100% RAM

Post by taashee »

Hello team ,

The database you are seeing is older database which is as it is untouched after doing the db offloading . The database was offloaded later on it's not from starting .
dchurch
Posts: 858
Joined: Wed Oct 07, 2020 12:46 pm
Location: Yo mama

Re: Nagios running slowly and utilizing 100% RAM

Post by dchurch »

Your profile states "localhost" for all three of the mysql databases, nagiosxi, ndoutils, and nagiosql:

Code: Select all

$cfg['db_info']=array(
	"nagiosxi" => array(
		"dbtype" => 'mysql',
		"dbserver" => 'localhost',
		"user" => 'nagiosxi',
		"db" => 'nagiosxi',
	"ndoutils" => array(
		"dbtype" => 'mysql',
		"dbserver" => 'localhost',
		"user" => 'ndoutils',
		"db" => 'nagios',
	"nagiosql" => array(
		"dbtype" => 'mysql',
		"dbserver" => 'localhost',
		"user" => 'nagiosql',
		"db" => 'nagiosql',
	);
Perhaps the profile you pulled was from a different server from the Nagios XI install you're having problems with. Did you install Nagios XI on two different servers, and are you using one as the offloaded database host?

The profile also shows MySQL using CPU cycles, so there's activity on it. Whether that's from internal or external connections, something is using that database.
If you didn't get an 8% raise over the course of the pandemic, you took a pay cut.

Discussion of wages is protected speech under the National Labor Relations Act, and no employer can tell you you can't disclose your pay with your fellow employees.
Locked