Page 3 of 3
Re: Getting frequent Swap and Memory warning/critical messag
Posted: Wed Nov 18, 2020 2:00 am
by pratikmehta003
Hi Benjamin,
tried to send via Private message but i guess it didnt go thru..attaching here..
And i got the RAM increased on the server but still see it fluctuating... i also got an alert for Load parameter yest.. and whenever memory and swap is getting reduced, monitoring engine process shows in Red.. when i start it from console, i see the alerts and notifications get updated so which means events are also getting stuck....
Could it also be due to the checks being done? in this customer they want an alert for bandwidth utilization only after 45 min or so.. below screenshot for what is currently configured
checks screenshot.PNG
Moderator's Note: The profile has been shared with the support team but has been removed from the public forum.
Re: Getting frequent Swap and Memory warning/critical messag
Posted: Wed Nov 18, 2020 3:55 pm
by benjaminsmith
Hi
@pratikmehta003
Thanks for getting the profile over. So there are corrupted tables in the database.
Code: Select all
201118 6:20:40 [ERROR] mysqld: Table './nagios/nagios_systemcommands' is marked as crashed and last (automatic?) repair failed
201118 6:20:40 [ERROR] mysqld: Table './nagios/nagios_systemcommands' is marked as crashed and last (automatic?) repair failed
This is causing connection issues with the backend database program, and this would result in delayed status updates in the GUI.
Code: Select all
[1605680440] NDO-3: ndo_handle_system_command(ndo-handlers.c:271): Could not reconnect to MySQL database
[1605680440] NDO-3: Unable to prepare statement for query (11): Table './nagios/nagios_systemcommands' is marked as crashed and last (automatic?) repair failed
Please log in as root the terminal, and run the following command to repair the database. If you experience any errors, please post them to the thread. If it's successful, let me know if the Nagios engine is still stopping or not and if you notice any performance improvements.
Code: Select all
/usr/local/nagiosxi/scripts/repair_databases.sh
Re: Getting frequent Swap and Memory warning/critical messag
Posted: Wed Nov 18, 2020 4:44 pm
by ssax
In addition to what @benjaminsmith posted (please follow his instructions).
Once you're repaired your databases, please send the output of this command:
- NOTE: You may need to adjust the -h 127.0.0.1, the -uroot, and -pnagiosxi in the command if your DB is offloaded to another server and/or you've changed the root mysql password
Code: Select all
echo "SELECT table_name AS 'Table', round(((data_length + index_length) / 1024 / 1024), 2) 'Size in MB' FROM information_schema.TABLES WHERE table_schema IN ('nagios', 'nagiosql', 'nagiosxi');" | mysql -h 127.0.0.1 -uroot -pnagiosxi --table
Re: Getting frequent Swap and Memory warning/critical messag
Posted: Thu Nov 19, 2020 1:07 pm
by pratikmehta003
Hi Benjamin,
Sure, i will run that repair command and let u know.. i will wait for the backup and post that i wil run this..
Re: Getting frequent Swap and Memory warning/critical messag
Posted: Fri Nov 20, 2020 2:07 pm
by benjaminsmith
Hi,
Sure, i will run that repair command and let u know.
Sounds good. We'll wait for your reply.
Re: Getting frequent Swap and Memory warning/critical messag
Posted: Wed Nov 25, 2020 1:26 am
by pratikmehta003
Hi Benjamin,
I ran the repair command it gave Success and the end.. attached the output of the command u requested..
Re: Getting frequent Swap and Memory warning/critical messag
Posted: Wed Nov 25, 2020 2:58 pm
by benjaminsmith
Hi,
That output looks normal. What is the current status? I believe most of the issues were the result of the corrupted database tables.
Is the monitoring engine is still stopping and how often? Please let me know. If this is occurring frequently, let's move this over to a support ticket, and as we may need to book a remote session to further troubleshoot.
To open a support ticket, please go to:
https://support.nagios.com/tickets/
Otherwise, please send me a fresh profile and I'll check the logs since making the database repair.
Best Regards,
Benjamin
Re: Getting frequent Swap and Memory warning/critical messag
Posted: Thu Nov 26, 2020 10:45 am
by pratikmehta003
Hi Benjamin,
From yest looks stable..let me monitor for day and will confirm.
Re: Getting frequent Swap and Memory warning/critical messag
Posted: Mon Nov 30, 2020 4:03 pm
by benjaminsmith
Hi,
From yest looks stable..let me monitor for day and will confirm.
Sounds good. We can leave this thread open, and just update it if you need anything.