Notification History Issue

[rajasegar]

Maybe it is a corrupt MYSQL entry causing it.
Do you see any errors in the /var/log/mysqld.log file?
Can you create a new copy of that check and see if that works for you?
Also, run the following as root on the server and post the output here.

Code: Select all

ps -ef --cols=300

No errors in mysqld.log

ps -ef --cols=300

ps_log.zip

Mysql check results

Code: Select all

Enter password:
mysql.columns_priv                                 OK
mysql.db                                           OK
mysql.event                                        OK
mysql.func                                         OK
mysql.general_log
Error    : You can't use locks with log tables.
status   : OK
mysql.help_category                                OK
mysql.help_keyword                                 OK
mysql.help_relation                                OK
mysql.help_topic                                   OK
mysql.host                                         OK
mysql.ndb_binlog_index                             OK
mysql.plugin                                       OK
mysql.proc                                         OK
mysql.procs_priv                                   OK
mysql.servers                                      OK
mysql.slow_log
Error    : You can't use locks with log tables.
status   : OK
mysql.tables_priv                                  OK
mysql.time_zone                                    OK
mysql.time_zone_leap_second                        OK
mysql.time_zone_name                               OK
mysql.time_zone_transition                         OK
mysql.time_zone_transition_type                    OK
mysql.user                                         OK
nagios.nagios_acknowledgements                     OK
nagios.nagios_commands                             OK
nagios.nagios_commenthistory                       OK
nagios.nagios_comments                             OK
nagios.nagios_configfiles                          OK
nagios.nagios_configfilevariables                  OK
nagios.nagios_conninfo                             OK
nagios.nagios_contact_addresses                    OK
nagios.nagios_contact_notificationcommands         OK
nagios.nagios_contactgroup_members                 OK
nagios.nagios_contactgroups                        OK
nagios.nagios_contactnotificationmethods           OK
nagios.nagios_contactnotifications                 OK
nagios.nagios_contacts                             OK
nagios.nagios_contactstatus                        OK
nagios.nagios_customvariables                      OK
nagios.nagios_customvariablestatus                 OK
nagios.nagios_dbversion                            OK
nagios.nagios_downtimehistory                      OK
nagios.nagios_eventhandlers                        OK
nagios.nagios_externalcommands                     OK
nagios.nagios_flappinghistory                      OK
nagios.nagios_host_contactgroups                   OK
nagios.nagios_host_contacts                        OK
nagios.nagios_host_parenthosts                     OK
nagios.nagios_hostchecks                           OK
nagios.nagios_hostdependencies                     OK
nagios.nagios_hostescalation_contactgroups         OK
nagios.nagios_hostescalation_contacts              OK
nagios.nagios_hostescalations                      OK
nagios.nagios_hostgroup_members                    OK
nagios.nagios_hostgroups                           OK
nagios.nagios_hosts                                OK
nagios.nagios_hoststatus                           OK
nagios.nagios_instances                            OK
nagios.nagios_logentries                           OK
nagios.nagios_notifications                        OK
nagios.nagios_objects                              OK
nagios.nagios_processevents                        OK
nagios.nagios_programstatus                        OK
nagios.nagios_runtimevariables                     OK
nagios.nagios_scheduleddowntime                    OK
nagios.nagios_service_contactgroups                OK
nagios.nagios_service_contacts                     OK
nagios.nagios_service_parentservices               OK
nagios.nagios_servicechecks                        OK
nagios.nagios_servicedependencies                  OK
nagios.nagios_serviceescalation_contactgroups      OK
nagios.nagios_serviceescalation_contacts           OK
nagios.nagios_serviceescalations                   OK
nagios.nagios_servicegroup_members                 OK
nagios.nagios_servicegroups                        OK
nagios.nagios_services                             OK
nagios.nagios_servicestatus                        OK
nagios.nagios_statehistory                         OK
nagios.nagios_systemcommands                       OK
nagios.nagios_timedeventqueue                      OK
nagios.nagios_timedevents                          OK
nagios.nagios_timeperiod_timeranges                OK
nagios.nagios_timeperiods                          OK
nagiosql.tbl_command                               OK
nagiosql.tbl_contact                               OK
nagiosql.tbl_contactgroup                          OK
nagiosql.tbl_contacttemplate                       OK
nagiosql.tbl_domain                                OK
nagiosql.tbl_host                                  OK
nagiosql.tbl_hostdependency                        OK
nagiosql.tbl_hostescalation                        OK
nagiosql.tbl_hostextinfo                           OK
nagiosql.tbl_hostgroup                             OK
nagiosql.tbl_hosttemplate                          OK
nagiosql.tbl_info                                  OK
nagiosql.tbl_lnkContactToCommandHost               OK
nagiosql.tbl_lnkContactToCommandService            OK
nagiosql.tbl_lnkContactToContactgroup              OK
nagiosql.tbl_lnkContactToContacttemplate           OK
nagiosql.tbl_lnkContactToVariabledefinition        OK
nagiosql.tbl_lnkContactgroupToContact              OK
nagiosql.tbl_lnkContactgroupToContactgroup         OK
nagiosql.tbl_lnkContacttemplateToCommandHost       OK
nagiosql.tbl_lnkContacttemplateToCommandService    OK
nagiosql.tbl_lnkContacttemplateToContactgroup      OK
nagiosql.tbl_lnkContacttemplateToContacttemplate   OK
nagiosql.tbl_lnkContacttemplateToVariabledefinition OK
nagiosql.tbl_lnkHostToContact                      OK
nagiosql.tbl_lnkHostToContactgroup                 OK
nagiosql.tbl_lnkHostToHost                         OK
nagiosql.tbl_lnkHostToHostgroup                    OK
nagiosql.tbl_lnkHostToHosttemplate                 OK
nagiosql.tbl_lnkHostToVariabledefinition           OK
nagiosql.tbl_lnkHostdependencyToHost_DH            OK
nagiosql.tbl_lnkHostdependencyToHost_H             OK
nagiosql.tbl_lnkHostdependencyToHostgroup_DH       OK
nagiosql.tbl_lnkHostdependencyToHostgroup_H        OK
nagiosql.tbl_lnkHostescalationToContact            OK
nagiosql.tbl_lnkHostescalationToContactgroup       OK
nagiosql.tbl_lnkHostescalationToHost               OK
nagiosql.tbl_lnkHostescalationToHostgroup          OK
nagiosql.tbl_lnkHostgroupToHost                    OK
nagiosql.tbl_lnkHostgroupToHostgroup               OK
nagiosql.tbl_lnkHosttemplateToContact              OK
nagiosql.tbl_lnkHosttemplateToContactgroup         OK
nagiosql.tbl_lnkHosttemplateToHost                 OK
nagiosql.tbl_lnkHosttemplateToHostgroup            OK
nagiosql.tbl_lnkHosttemplateToHosttemplate         OK
nagiosql.tbl_lnkHosttemplateToVariabledefinition   OK
nagiosql.tbl_lnkServiceToContact                   OK
nagiosql.tbl_lnkServiceToContactgroup              OK
nagiosql.tbl_lnkServiceToHost                      OK
nagiosql.tbl_lnkServiceToHostgroup                 OK
nagiosql.tbl_lnkServiceToServicegroup              OK
nagiosql.tbl_lnkServiceToServicetemplate           OK
nagiosql.tbl_lnkServiceToVariabledefinition        OK
nagiosql.tbl_lnkServicedependencyToHost_DH         OK
nagiosql.tbl_lnkServicedependencyToHost_H          OK
nagiosql.tbl_lnkServicedependencyToHostgroup_DH    OK
nagiosql.tbl_lnkServicedependencyToHostgroup_H     OK
nagiosql.tbl_lnkServicedependencyToService_DS      OK
nagiosql.tbl_lnkServicedependencyToService_S       OK
nagiosql.tbl_lnkServiceescalationToContact         OK
nagiosql.tbl_lnkServiceescalationToContactgroup    OK
nagiosql.tbl_lnkServiceescalationToHost            OK
nagiosql.tbl_lnkServiceescalationToHostgroup       OK
nagiosql.tbl_lnkServiceescalationToService         OK
nagiosql.tbl_lnkServicegroupToService              OK
nagiosql.tbl_lnkServicegroupToServicegroup         OK
nagiosql.tbl_lnkServicetemplateToContact           OK
nagiosql.tbl_lnkServicetemplateToContactgroup      OK
nagiosql.tbl_lnkServicetemplateToHost              OK
nagiosql.tbl_lnkServicetemplateToHostgroup         OK
nagiosql.tbl_lnkServicetemplateToServicegroup      OK
nagiosql.tbl_lnkServicetemplateToServicetemplate   OK
nagiosql.tbl_lnkServicetemplateToVariabledefinition OK
nagiosql.tbl_lnkTimeperiodToTimeperiod             OK
nagiosql.tbl_logbook                               OK
nagiosql.tbl_mainmenu                              OK
nagiosql.tbl_service                               OK
nagiosql.tbl_servicedependency                     OK
nagiosql.tbl_serviceescalation                     OK
nagiosql.tbl_serviceextinfo                        OK
nagiosql.tbl_servicegroup                          OK
nagiosql.tbl_servicetemplate                       OK
nagiosql.tbl_settings                              OK
nagiosql.tbl_submenu                               OK
nagiosql.tbl_timedefinition                        OK
nagiosql.tbl_timeperiod                            OK
nagiosql.tbl_user                                  OK
nagiosql.tbl_variabledefinition                    OK
You have mail in /var/spool/mail/nagios

[tgriep]

Did you try and create a new copy of that service check and see if that fixes it for you?

[rajasegar]

Did you try and create a new copy of that service check and see if that fixes it for you?

There is 2 hosts attached to this service.
1 is working fine and the other does not show notification history.
This is the perfect scenario for troubleshooting.

If I delete the service and it becomes ok, how are you going to determine what was wrong and prevent it from happening in future?

[tgriep]

Without knowing how to fix the issue, we wont know where to look to figure out what caused it even if that is possible.
If you make a new copy of that check but keep the old one in the system deactivated, and the new one works, then it could be a corruption in the MYSQL database.

[rajasegar]

Without knowing how to fix the issue, we wont know where to look to figure out what caused it even if that is possible.
If you make a new copy of that check but keep the old one in the system deactivated, and the new one works, then it could be a corruption in the MYSQL database.

I have duplicated the service, will update later if there is any status changes.

[tgriep]

Thanks. Let us know how it works out.

[rajasegar]

Thanks. Let us know how it works out.

Same issue with the duplicated service. No notification history

[tgriep]

Can you post the email that you are receiving for this service?