Page 1 of 2
Nagiosxi Corruption
Posted: Mon Oct 31, 2016 9:53 am
by sanjayhiranisq
We have an issue where nagiosxi server was shutdown ( cleanly) but after some configuration the disk subsystem was very slow ( but accessible). Nagios asked for the db recovery scripts to be run which we did and now we have some corruption in XI.
So using nagios classic we can see all objects and alerts are working and metrics are showing. we can see service groups status and host status etc
In nagiosxi we are able to see the objects in core config manager, but not in the frontend. no objects are shown no servuce groups, hostgroups etc. The dashboards that we have with graphs and other netrics are working fine.
Please could you help
Re: Nagiosxi Corruption
Posted: Mon Oct 31, 2016 11:23 am
by rkennedy
Could you show us a screenshot outlining what you're seeing on your frontend display currently?
Also, what version of Nagios XI are you running?
Re: Nagiosxi Corruption
Posted: Mon Oct 31, 2016 11:50 am
by sanjayhiranisq
Hi,
I have attached screenshots depicting the issue.
Thank you
Sanjay
Re: Nagiosxi Corruption
Posted: Mon Oct 31, 2016 12:28 pm
by avandemore
XI > Admin > System Profile > Download Profile
Please include the zip file in your response. You can PM myself or other support personnel if there are privacy concerns.
Re: Nagiosxi Corruption
Posted: Wed Nov 02, 2016 6:02 am
by sanjayhiranisq
profile sent via pm
Re: Nagiosxi Corruption
Posted: Wed Nov 02, 2016 6:54 am
by sanjayhiranisq
The attached image shows an interesting output for engine status
Re: Nagiosxi Corruption
Posted: Wed Nov 02, 2016 9:33 am
by avandemore
Profile received.
First thing is your Apache error log is filled with these:
Code: Select all
[client 10.64.2.39] user siegerje: authentication failure for "/nagios/cgi-bin/status.cgi": Password Mismatch
Something to keep an eye on, possibility related.
Please post the output of this:
Code: Select all
mysqlcheck -r -f -u root -pnagiosxi --databases nagios
Perf data is also stalling due to high load.
Re: Nagiosxi Corruption
Posted: Wed Nov 02, 2016 11:21 am
by sanjayhiranisq
EDIT: Please use code tag for long output
Code: Select all
[2016-11-02 16:19:28 root@nyzls114i ~]# mysqlcheck -r -f -u root -pnagiosxi --databases nagios
nagios.nagios_acknowledgements OK
nagios.nagios_commands OK
nagios.nagios_commenthistory OK
nagios.nagios_comments OK
nagios.nagios_configfiles OK
nagios.nagios_configfilevariables OK
nagios.nagios_conninfo OK
nagios.nagios_contact_addresses OK
nagios.nagios_contact_notificationcommands OK
nagios.nagios_contactgroup_members OK
nagios.nagios_contactgroups OK
nagios.nagios_contactnotificationmethods OK
nagios.nagios_contactnotifications OK
nagios.nagios_contacts OK
nagios.nagios_contactstatus OK
nagios.nagios_customvariables OK
nagios.nagios_customvariablestatus OK
nagios.nagios_dbversion OK
nagios.nagios_downtimehistory OK
nagios.nagios_eventhandlers OK
nagios.nagios_externalcommands OK
nagios.nagios_flappinghistory OK
nagios.nagios_host_contactgroups OK
nagios.nagios_host_contacts OK
nagios.nagios_host_parenthosts OK
nagios.nagios_hostchecks OK
nagios.nagios_hostdependencies OK
nagios.nagios_hostescalation_contactgroups OK
nagios.nagios_hostescalation_contacts OK
nagios.nagios_hostescalations OK
nagios.nagios_hostgroup_members OK
nagios.nagios_hostgroups OK
nagios.nagios_hosts OK
nagios.nagios_hoststatus OK
nagios.nagios_instances OK
nagios.nagios_logentries OK
nagios.nagios_notifications OK
nagios.nagios_objects OK
nagios.nagios_processevents OK
nagios.nagios_programstatus OK
nagios.nagios_runtimevariables OK
nagios.nagios_scheduleddowntime OK
nagios.nagios_service_contactgroups OK
nagios.nagios_service_contacts OK
nagios.nagios_service_parentservices OK
nagios.nagios_servicechecks OK
nagios.nagios_servicedependencies OK
nagios.nagios_serviceescalation_contactgroups OK
nagios.nagios_serviceescalation_contacts OK
nagios.nagios_serviceescalations OK
nagios.nagios_servicegroup_members OK
nagios.nagios_servicegroups OK
nagios.nagios_services OK
nagios.nagios_servicestatus OK
nagios.nagios_statehistory OK
nagios.nagios_systemcommands OK
nagios.nagios_timedeventqueue OK
nagios.nagios_timedevents OK
nagios.nagios_timeperiod_timeranges OK
nagios.nagios_timeperiods OK
[2016-11-02 16:20:53 root@nyzls114i ~]#
Re: Nagiosxi Corruption
Posted: Wed Nov 02, 2016 11:37 am
by avandemore
XI > Configure > Core Config Manager > Config File Management > Delete Files > Write Configs > Verify Files > Restart Nagios Core
If that is too slow, try this instead:
/usr/local/nagios/bin/nagios -vvv /usr/local/nagios/etc/nagios.cfg
Wait about 10 minutes and let me know if the Monitoring status changes. Also if there were any errors in this process let me know.
This document should also be tried. I don't think you'll get any output, but let's see.
Repairing The Database
Re: Nagiosxi Corruption
Posted: Wed Nov 02, 2016 11:53 am
by sanjayhiranisq
Followed the above steps and the database repair and there is no change. I am wondering if there is a corrupt script somewhere. Do you have the cksums of all the important files for the latest version?