Nagios Support Forum

began seeing this error again:

2012-08-14T10:27:41.313565-04:00 id-nag01 ndo2db: Error: mysql_query() failed for 'INSERT INTO nagios_notifications SET instance_id='1', notification_ty
pe='1', notification_reason='0', start_time=FROM_UNIXTIME(1344953061), start_time_usec='148075', end_time=FROM_UNIXTIME(0), end_time_usec='0', object_id
='6056', state='2', output='Connection refused by host', long_output='Connection refused by host', escalated='0', contacts_notified='0' ON DUPLICATE KEY
UPDATE instance_id='1', notification_type='1', notification_reason='0', start_time=FROM_UNIXTIME(1344953061), start_time_usec='148075', end_time=FROM_U
NIXTIME(0), end_time_usec='0', object_id='6056', state='2', output='Connection refused by host', long_output='Connection refused by host', escalated='0'
, contacts_notified='0''
2012-08-14T10:27:41.313584-04:00 id-nag01 ndo2db: mysql_error: 'Server shutdown in progress'
2012-08-14T10:27:41.313764-04:00 id-nag01 ndo2db: Error: mysql_query() failed for 'DELETE FROM nagios_timedevents WHERE instance_id='1' AND scheduled_ti
me<FROM_UNIXTIME(1344868061)'
2012-08-14T10:27:41.313772-04:00 id-nag01 ndo2db: mysql_error: 'Lost connection to MySQL server during query'
2012-08-14T10:27:41.313778-04:00 id-nag01 ndo2db: Error: Connection to MySQL database has been lost!

running again now after restarting all services.

These logs are only showing information during s mysql shutdown. The logs you posted only span a time period of 0.0002 seconds.

the last refresh time in the xi interface was approximately 10:04AM. I see nothing prior in /var/log/messages until 10:27 at which point we see the mysql errors start to flow in.

is there another error log I should be looking for?

looking now there is a 10 minute gap in /var/log/messages, starting with an queue send error for ndo2db:

2012-08-14T10:17:09.318123-04:00 id-nag01 ndo2db: Error: queue send error, retrying...

2012-08-14T10:17:09.320375-04:00 id-nag01 nagios: SERVICE NOTIFICATION: nagiosadmin;id-ufm01b;SSHD Process;CRITICAL;xi_service_notification_handler;Conn
ection refused by host
2012-08-14T10:17:09.457583-04:00 id-nag01 nagios: SERVICE NOTIFICATION: nagiosadmin;pd-cpm02;Memory Usage;WARNING;xi_service_notification_handler;WARNING - 356 / 24151 MB (1%) Free Memory, Used: 23795 MB, Shared: 0 MB, Buffers: 553 MB, Cached: 15049 MB
2012-08-14T10:27:14.155966-04:00 id-nag01 nscd: nss_ldap: reconnected to LDAP server ldap://11.4.8.81
2012-08-14T10:27:41.313565-04:00 id-nag01 ndo2db: Error: mysql_query() failed for 'INSERT INTO nagios_notifications SET instance_id='1', notification_ty
pe='1', notification_reason='0', start_time=FROM_UNIXTIME(1344953061), start_time_usec='148075', end_time=FROM_UNIXTIME(0), end_time_usec='0', object_id
='6056', state='2', output='Connection refused by host', long_output='Connection refused by host', escalated='0', contacts_notified='0' ON DUPLICATE KEY
UPDATE instance_id='1', notification_type='1', notification_reason='0', start_time=FROM_UNIXTIME(1344953061), start_time_usec='148075', end_time=FROM_U
NIXTIME(0), end_time_usec='0', object_id='6056', state='2', output='Connection refused by host', long_output='Connection refused by host', escalated='0'
, contacts_notified='0''
2012-08-14T10:27:41.313584-04:00 id-nag01 ndo2db: mysql_error: 'Server shutdown in progress'
2012-08-14T10:27:41.313764-04:00 id-nag01 ndo2db: Error: mysql_query() failed for 'DELETE FROM nagios_timedevents WHERE instance_id='1' AND scheduled_ti
me<FROM_UNIXTIME(1344868061)'

The error you are seeing here Is only happening because mysql had already initiated the shutdown which was logged at To have an occasional line like the following is normal If you have a stream of them or a message saying queue items are deleted, that would be a different story.