Getting frequent Swap and Memory warning/critical message

[pratikmehta003]

Hi Benjamin,

tried to send via Private message but i guess it didnt go thru..attaching here..

And i got the RAM increased on the server but still see it fluctuating... i also got an alert for Load parameter yest.. and whenever memory and swap is getting reduced, monitoring engine process shows in Red.. when i start it from console, i see the alerts and notifications get updated so which means events are also getting stuck....

Could it also be due to the checks being done? in this customer they want an alert for bandwidth utilization only after 45 min or so.. below screenshot for what is currently configured

checks screenshot.PNG

Moderator's Note: The profile has been shared with the support team but has been removed from the public forum.

[benjaminsmith]

Hi @pratikmehta003

Thanks for getting the profile over. So there are corrupted tables in the database.

Code: Select all

201118  6:20:40 [ERROR] mysqld: Table './nagios/nagios_systemcommands' is marked as crashed and last (automatic?) repair failed
201118  6:20:40 [ERROR] mysqld: Table './nagios/nagios_systemcommands' is marked as crashed and last (automatic?) repair failed

This is causing connection issues with the backend database program, and this would result in delayed status updates in the GUI.

Code: Select all

[1605680440] NDO-3: ndo_handle_system_command(ndo-handlers.c:271): Could not reconnect to MySQL database
[1605680440] NDO-3: Unable to prepare statement for query (11): Table './nagios/nagios_systemcommands' is marked as crashed and last (automatic?) repair failed

Please log in as root the terminal, and run the following command to repair the database. If you experience any errors, please post them to the thread. If it's successful, let me know if the Nagios engine is still stopping or not and if you notice any performance improvements.

Code: Select all

/usr/local/nagiosxi/scripts/repair_databases.sh

[ssax]

In addition to what @benjaminsmith posted (please follow his instructions).

Once you're repaired your databases, please send the output of this command:
- NOTE: You may need to adjust the -h 127.0.0.1, the -uroot, and -pnagiosxi in the command if your DB is offloaded to another server and/or you've changed the root mysql password

Code: Select all

echo "SELECT table_name AS 'Table', round(((data_length + index_length) / 1024 / 1024), 2) 'Size in MB' FROM information_schema.TABLES WHERE table_schema IN ('nagios', 'nagiosql', 'nagiosxi');" | mysql -h 127.0.0.1 -uroot -pnagiosxi --table

[pratikmehta003]

Hi Benjamin,

Sure, i will run that repair command and let u know.. i will wait for the backup and post that i wil run this..

[benjaminsmith]

Hi,

Sure, i will run that repair command and let u know.

Sounds good. We'll wait for your reply.

[pratikmehta003]

Hi Benjamin,

I ran the repair command it gave Success and the end.. attached the output of the command u requested..

[benjaminsmith]

Hi,

That output looks normal. What is the current status? I believe most of the issues were the result of the corrupted database tables.

Is the monitoring engine is still stopping and how often? Please let me know. If this is occurring frequently, let's move this over to a support ticket, and as we may need to book a remote session to further troubleshoot.

To open a support ticket, please go to:
https://support.nagios.com/tickets/

Otherwise, please send me a fresh profile and I'll check the logs since making the database repair.

Best Regards,
Benjamin

[pratikmehta003]

Hi Benjamin,

From yest looks stable..let me monitor for day and will confirm.

[benjaminsmith]

Hi,

From yest looks stable..let me monitor for day and will confirm.

Sounds good. We can leave this thread open, and just update it if you need anything.