System Component Status are Red

[ITOMB_IMT]

i see a user has issues when he logs into Xi. he cant see dashboards/anything except sql errors, when refreshed they disappears.

see below screenshot.

[cdienger]

There's an issue with the offloaded database. Are you aware of any problems with it? Can you provide the mysql logs from that machine?

[ITOMB_IMT]

I am not aware of the problems with offloaded database. I sent a pm with offloaded database. Please have a look.


Thanks,

[cdienger]

The logs show a problem with replication including failure to read nagios tables starting on the 15th. These should be addressed by the DBA - can you disable replication for the nagios, nagiosxi, and nagiosql databases?

[ITOMB_IMT]

Even before Replication exists and after the successful replication , the user is seeing those errors.

[cdienger]

These are the only errors I've really seen in the data that has been provided and given the behavior in the GUI seem like a valid thing to check into.

[ITOMB_IMT]

Please see the PM attached with latest profile.

[cdienger]

Please also provide a copy of the sql logs as well.

I'd also like to see the output of:

Code: Select all

echo "SELECT table_name AS 'Table', round(((data_length + index_length) / 1024 / 1024), 2) 'Size in MB' FROM information_schema.TABLES WHERE table_schema IN ('nagios');" | mysql -h REMOTEDBIP -uUSERNAME -pPASSWORD -Dnagios --table
echo "SELECT table_name AS 'Table', round(((data_length + index_length) / 1024 / 1024), 2) 'Size in MB' FROM information_schema.TABLES WHERE table_schema IN ('nagiosql');" | mysql -h REMOTEDBIP -uUSERNAME -pPASSWORD -Dnagiosql --table
echo "SELECT table_name AS 'Table', round(((data_length + index_length) / 1024 / 1024), 2) 'Size in MB' FROM information_schema.TABLES WHERE table_schema IN ('nagiosxi');" | mysql -h REMOTEDBIP -uUSERNAME -pPASSWORD -Dnagiosxi --table

[ITOMB_IMT]

see the output of all databases, and also i sent you the pm of db logs.

echo "SELECT table_name AS 'Table', round(((data_length + index_length) / 1024 / 1024), 2) 'Size in MB' FROM information_schema.TABLES WHERE table_schema IN ('nagios');" | mysql -h REMOTEDBIP -uUSERNAME -pPASSWORD -Dnagios --table
+----------------------------------------+------------+
| Table | Size in MB |
+----------------------------------------+------------+
| nagios_acknowledgements | 0.00 |
| nagios_commands | 0.02 |
| nagios_commenthistory | 39.99 |
| nagios_comments | 0.02 |
| nagios_configfiles | 0.00 |
| nagios_configfilevariables | 0.01 |
| nagios_conninfo | 0.33 |
| nagios_contact_addresses | 0.00 |
| nagios_contact_notificationcommands | 0.02 |
| nagios_contactgroup_members | 0.00 |
| nagios_contactgroups | 0.00 |
| nagios_contactnotificationmethods | 0.42 |
| nagios_contactnotifications | 0.45 |
| nagios_contacts | 0.00 |
| nagios_contactstatus | 0.00 |
| nagios_customvariables | 0.14 |
| nagios_customvariablestatus | 0.14 |
| nagios_dbversion | 0.00 |
| nagios_downtimehistory | 0.13 |
| nagios_eventhandlers | 0.00 |
| nagios_externalcommands | 0.01 |
| nagios_flappinghistory | 0.02 |
| nagios_host_contactgroups | 0.00 |
| nagios_host_contacts | 0.01 |
| nagios_host_parenthosts | 0.00 |
| nagios_hostchecks | 0.00 |
| nagios_hostdependencies | 0.00 |
| nagios_hostescalation_contactgroups | 0.00 |
| nagios_hostescalation_contacts | 0.00 |
| nagios_hostescalations | 0.00 |
| nagios_hostgroup_members | 0.00 |
| nagios_hostgroups | 0.00 |
| nagios_hosts | 0.03 |
| nagios_hoststatus | 0.05 |
| nagios_instances | 0.00 |
| nagios_logentries | 93.43 |
| nagios_notifications | 1.46 |
| nagios_objects | 0.74 |
| nagios_processevents | 0.72 |
| nagios_programstatus | 0.00 |
| nagios_runtimevariables | 0.00 |
| nagios_scheduleddowntime | 0.01 |
| nagios_service_contactgroups | 0.02 |
| nagios_service_contacts | 0.10 |
| nagios_service_parentservices | 0.00 |
| nagios_servicechecks | 0.00 |
| nagios_servicedependencies | 0.00 |
| nagios_serviceescalation_contactgroups | 0.00 |
| nagios_serviceescalation_contacts | 0.00 |
| nagios_serviceescalations | 0.00 |
| nagios_servicegroup_members | 0.00 |
| nagios_servicegroups | 0.00 |
| nagios_services | 0.29 |
| nagios_servicestatus | 0.76 |
| nagios_statehistory | 1.83 |
| nagios_systemcommands | 0.04 |
| nagios_timedeventqueue | 0.00 |
| nagios_timedevents | 0.00 |
| nagios_timeperiod_timeranges | 0.01 |
| nagios_timeperiods | 0.00 |
+----------------------------------------+------------+
# echo "SELECT table_name AS 'Table', round(((data_length + index_length) / 1024 / 1024), 2) 'Size in MB' FROM information_schema.TABLES WHERE table_schema IN ('nagiosql');" | mysql -h REMOTEDBIP -uUSERNAME -pPASSWORD -Dnagiosql --table
+--------------------------------------------+------------+
| Table | Size in MB |
+--------------------------------------------+------------+
| tbl_command | 0.06 |
| tbl_contact | 0.03 |
| tbl_contactgroup | 0.03 |
| tbl_contacttemplate | 0.03 |
| tbl_domain | 0.03 |
| tbl_host | 0.03 |
| tbl_hostdependency | 0.03 |
| tbl_hostescalation | 0.03 |
| tbl_hostextinfo | 0.03 |
| tbl_hostgroup | 0.03 |
| tbl_hosttemplate | 0.03 |
| tbl_info | 0.17 |
| tbl_lnkContactToCommandHost | 0.02 |
| tbl_lnkContactToCommandService | 0.02 |
| tbl_lnkContactToContactgroup | 0.02 |
| tbl_lnkContactToContacttemplate | 0.02 |
| tbl_lnkContactToVariabledefinition | 0.02 |
| tbl_lnkContactgroupToContact | 0.02 |
| tbl_lnkContactgroupToContactgroup | 0.02 |
| tbl_lnkContacttemplateToCommandHost | 0.02 |
| tbl_lnkContacttemplateToCommandService | 0.02 |
| tbl_lnkContacttemplateToContactgroup | 0.02 |
| tbl_lnkContacttemplateToContacttemplate | 0.02 |
| tbl_lnkContacttemplateToVariabledefinition | 0.02 |
| tbl_lnkHostToContact | 0.02 |
| tbl_lnkHostToContactgroup | 0.02 |
| tbl_lnkHostToHost | 0.02 |
| tbl_lnkHostToHostgroup | 0.02 |
| tbl_lnkHostToHosttemplate | 0.02 |
| tbl_lnkHostToVariabledefinition | 0.02 |
| tbl_lnkHostdependencyToHost_DH | 0.02 |
| tbl_lnkHostdependencyToHost_H | 0.02 |
| tbl_lnkHostdependencyToHostgroup_DH | 0.02 |
| tbl_lnkHostdependencyToHostgroup_H | 0.02 |
| tbl_lnkHostescalationToContact | 0.02 |
| tbl_lnkHostescalationToContactgroup | 0.02 |
| tbl_lnkHostescalationToHost | 0.02 |
| tbl_lnkHostescalationToHostgroup | 0.02 |
| tbl_lnkHostgroupToHost | 0.02 |
| tbl_lnkHostgroupToHostgroup | 0.02 |
| tbl_lnkHosttemplateToContact | 0.02 |
| tbl_lnkHosttemplateToContactgroup | 0.02 |
| tbl_lnkHosttemplateToHost | 0.02 |
| tbl_lnkHosttemplateToHostgroup | 0.02 |
| tbl_lnkHosttemplateToHosttemplate | 0.02 |
| tbl_lnkHosttemplateToVariabledefinition | 0.02 |
| tbl_lnkServiceToContact | 0.09 |
| tbl_lnkServiceToContactgroup | 0.02 |
| tbl_lnkServiceToHost | 0.08 |
| tbl_lnkServiceToHostgroup | 0.02 |
| tbl_lnkServiceToServicegroup | 0.02 |
| tbl_lnkServiceToServicetemplate | 0.08 |
| tbl_lnkServiceToVariabledefinition | 0.08 |
| tbl_lnkServicedependencyToHost_DH | 0.02 |
| tbl_lnkServicedependencyToHost_H | 0.02 |
| tbl_lnkServicedependencyToHostgroup_DH | 0.02 |
| tbl_lnkServicedependencyToHostgroup_H | 0.02 |
| tbl_lnkServicedependencyToService_DS | 0.02 |
| tbl_lnkServicedependencyToService_S | 0.02 |
| tbl_lnkServicedependencyToServicegroup_DS | 0.02 |
| tbl_lnkServicedependencyToServicegroup_S | 0.02 |
| tbl_lnkServiceescalationToContact | 0.02 |
| tbl_lnkServiceescalationToContactgroup | 0.02 |
| tbl_lnkServiceescalationToHost | 0.02 |
| tbl_lnkServiceescalationToHostgroup | 0.02 |
| tbl_lnkServiceescalationToService | 0.02 |
| tbl_lnkServiceescalationToServicegroup | 0.02 |
| tbl_lnkServicegroupToService | 0.02 |
| tbl_lnkServicegroupToServicegroup | 0.02 |
| tbl_lnkServicetemplateToContact | 0.02 |
| tbl_lnkServicetemplateToContactgroup | 0.02 |
| tbl_lnkServicetemplateToHost | 0.02 |
| tbl_lnkServicetemplateToHostgroup | 0.02 |
| tbl_lnkServicetemplateToServicegroup | 0.02 |
| tbl_lnkServicetemplateToServicetemplate | 0.02 |
| tbl_lnkServicetemplateToVariabledefinition | 0.02 |
| tbl_lnkTimeperiodToTimeperiod | 0.02 |
| tbl_logbook | 0.02 |
| tbl_mainmenu | 0.02 |
| tbl_permission | 0.02 |
| tbl_permission_inactive | 0.02 |
| tbl_service | 0.34 |
| tbl_servicedependency | 0.03 |
| tbl_serviceescalation | 0.03 |
| tbl_serviceextinfo | 0.03 |
| tbl_servicegroup | 0.03 |
| tbl_servicetemplate | 0.03 |
| tbl_session | 0.02 |
| tbl_session_locks | 0.02 |
| tbl_settings | 0.03 |
| tbl_submenu | 0.02 |
| tbl_timedefinition | 0.02 |
| tbl_timeperiod | 0.03 |
| tbl_user | 0.03 |
| tbl_variabledefinition | 0.13 |
+--------------------------------------------+------------+
#echo "SELECT table_name AS 'Table', round(((data_length + index_length) / 1024 / 1024), 2) 'Size in MB' FROM information_schema.TABLES WHERE table_schema IN ('nagiosxi');" | mysql -h REMOTEDBIP -uUSERNAME -pPASSWORD -Dnagiosxi --table
+---------------------+------------+
| Table | Size in MB |
+---------------------+------------+
| xi_auditlog | 3.56 |
| xi_auth_tokens | 0.23 |
| xi_cmp_trapdata | 0.03 |
| xi_cmp_trapdata_log | 0.03 |
| xi_commands | 0.02 |
| xi_eventqueue | 0.03 |
| xi_events | 0.05 |
| xi_incidents | 0.02 |
| xi_meta | 0.20 |
| xi_mibs | 0.05 |
| xi_options | 0.03 |
| xi_sessions | 0.03 |
| xi_sysstat | 0.03 |
| xi_usermeta | 0.33 |
| xi_users | 0.03 |
+---------------------+------------+

[cdienger]

A problem with the nagiosql database that would cause errors like this was fixed in 5.5.10:

https://assets.nagios.com/downloads/nag ... NGES-5.TXT
- Fixed MySQL nagiosql errors in cmdsubsys.log for regular users with limited CCM access [TPS#14045] -JO

I would recommend updating to the latest version to take advantage of this fix and others - https://assets.nagios.com/downloads/nag ... ctions.pdf