Page 2 of 2
Re: Nagios not sending out notifications
Posted: Fri Feb 09, 2018 3:15 pm
by jkinning
Got another one.
Code: Select all
***** Nagios XI Alert *****
Nagios has detected a problem with this service.
Notification Type: PROBLEM
Service: Nagios XI Jobs
Host: nagiosl1p
Address:
State: CRITICAL
Info:
Database Maintenance (dbmaint) stale (39110 seconds old), Database Maintenance (dbmaint) stale (39110 seconds old)
Date/Time: 2018-02-09 14:59:08
Re: Nagios not sending out notifications
Posted: Fri Feb 09, 2018 3:48 pm
by npolovenko
@jkinning, Ok, let's try this. First, stop the crond service with:
Next, manually remove the dbmaint.lock file
Code: Select all
rm /usr/local/nagiosxi/var/dbmaint.lock
Start the crond and monitor the log file for any errors:
Code: Select all
service crond start
tail -f /usr/local/nagiosxi/var/dbmaint.log
Re: Nagios not sending out notifications
Posted: Fri Feb 09, 2018 9:24 pm
by jkinning
Still getting these alerts
Code: Select all
***** Nagios XI Alert *****
Nagios has detected a problem with this service.
Notification Type: PROBLEM
Service: Nagios XI Jobs
Host: nagiosl1p
Address:
State: CRITICAL
Info:
Database Maintenance (dbmaint) stale (61894 seconds old), Database Maintenance (dbmaint) stale (61894 seconds old)
Date/Time: 2018-02-09 21:18:53
Last few lines from the log file
Code: Select all
SQL: OPTIMIZE TABLE nagios_contactnotifications
OPTIMIZING NDOUTILS TABLE: nagios_contacts
SQL: OPTIMIZE TABLE nagios_contacts
OPTIMIZING NDOUTILS TABLE: nagios_contactstatus
SQL: OPTIMIZE TABLE nagios_contactstatus
OPTIMIZING NDOUTILS TABLE: nagios_customvariables
SQL: OPTIMIZE TABLE nagios_customvariables
OPTIMIZING NDOUTILS TABLE: nagios_customvariablestatus
SQL: OPTIMIZE TABLE nagios_customvariablestatus
OPTIMIZING NDOUTILS TABLE: nagios_dbversion
SQL: OPTIMIZE TABLE nagios_dbversion
OPTIMIZING NDOUTILS TABLE: nagios_downtimehistory
SQL: OPTIMIZE TABLE nagios_downtimehistory
OPTIMIZING NDOUTILS TABLE: nagios_eventhandlers
SQL: OPTIMIZE TABLE nagios_eventhandlers
OPTIMIZING NDOUTILS TABLE: nagios_externalcommands
SQL: OPTIMIZE TABLE nagios_externalcommands
OPTIMIZING NDOUTILS TABLE: nagios_flappinghistory
SQL: OPTIMIZE TABLE nagios_flappinghistory
OPTIMIZING NDOUTILS TABLE: nagios_host_contactgroups
SQL: OPTIMIZE TABLE nagios_host_contactgroups
OPTIMIZING NDOUTILS TABLE: nagios_host_contacts
SQL: OPTIMIZE TABLE nagios_host_contacts
OPTIMIZING NDOUTILS TABLE: nagios_host_parenthosts
SQL: OPTIMIZE TABLE nagios_host_parenthosts
OPTIMIZING NDOUTILS TABLE: nagios_hostchecks
SQL: OPTIMIZE TABLE nagios_hostchecks
OPTIMIZING NDOUTILS TABLE: nagios_hostdependencies
SQL: OPTIMIZE TABLE nagios_hostdependencies
OPTIMIZING NDOUTILS TABLE: nagios_hostescalation_contactgroups
SQL: OPTIMIZE TABLE nagios_hostescalation_contactgroups
OPTIMIZING NDOUTILS TABLE: nagios_hostescalation_contacts
SQL: OPTIMIZE TABLE nagios_hostescalation_contacts
OPTIMIZING NDOUTILS TABLE: nagios_hostescalations
SQL: OPTIMIZE TABLE nagios_hostescalations
OPTIMIZING NDOUTILS TABLE: nagios_hostgroup_members
SQL: OPTIMIZE TABLE nagios_hostgroup_members
OPTIMIZING NDOUTILS TABLE: nagios_hostgroups
SQL: OPTIMIZE TABLE nagios_hostgroups
OPTIMIZING NDOUTILS TABLE: nagios_hosts
SQL: OPTIMIZE TABLE nagios_hosts
OPTIMIZING NDOUTILS TABLE: nagios_hoststatus
SQL: OPTIMIZE TABLE nagios_hoststatus
OPTIMIZING NDOUTILS TABLE: nagios_instances
SQL: OPTIMIZE TABLE nagios_instances
OPTIMIZING NDOUTILS TABLE: nagios_logentries
SQL: OPTIMIZE TABLE nagios_logentries
OPTIMIZING NDOUTILS TABLE: nagios_notifications
SQL: OPTIMIZE TABLE nagios_notifications
OPTIMIZING NDOUTILS TABLE: nagios_objects
SQL: OPTIMIZE TABLE nagios_objects
OPTIMIZING NDOUTILS TABLE: nagios_processevents
SQL: OPTIMIZE TABLE nagios_processevents
OPTIMIZING NDOUTILS TABLE: nagios_programstatus
SQL: OPTIMIZE TABLE nagios_programstatus
OPTIMIZING NDOUTILS TABLE: nagios_runtimevariables
SQL: OPTIMIZE TABLE nagios_runtimevariables
OPTIMIZING NDOUTILS TABLE: nagios_scheduleddowntime
SQL: OPTIMIZE TABLE nagios_scheduleddowntime
OPTIMIZING NDOUTILS TABLE: nagios_service_contactgroups
SQL: OPTIMIZE TABLE nagios_service_contactgroups
OPTIMIZING NDOUTILS TABLE: nagios_service_contacts
SQL: OPTIMIZE TABLE nagios_service_contacts
OPTIMIZING NDOUTILS TABLE: nagios_servicechecks
SQL: OPTIMIZE TABLE nagios_servicechecks
OPTIMIZING NDOUTILS TABLE: nagios_servicedependencies
SQL: OPTIMIZE TABLE nagios_servicedependencies
OPTIMIZING NDOUTILS TABLE: nagios_serviceescalation_contactgroups
SQL: OPTIMIZE TABLE nagios_serviceescalation_contactgroups
OPTIMIZING NDOUTILS TABLE: nagios_serviceescalation_contacts
SQL: OPTIMIZE TABLE nagios_serviceescalation_contacts
OPTIMIZING NDOUTILS TABLE: nagios_serviceescalations
SQL: OPTIMIZE TABLE nagios_serviceescalations
OPTIMIZING NDOUTILS TABLE: nagios_servicegroup_members
SQL: OPTIMIZE TABLE nagios_servicegroup_members
OPTIMIZING NDOUTILS TABLE: nagios_servicegroups
SQL: OPTIMIZE TABLE nagios_servicegroups
OPTIMIZING NDOUTILS TABLE: nagios_services
SQL: OPTIMIZE TABLE nagios_services
OPTIMIZING NDOUTILS TABLE: nagios_servicestatus
SQL: OPTIMIZE TABLE nagios_servicestatus
OPTIMIZING NDOUTILS TABLE: nagios_statehistory
SQL: OPTIMIZE TABLE nagios_statehistory
OPTIMIZING NDOUTILS TABLE: nagios_systemcommands
SQL: OPTIMIZE TABLE nagios_systemcommands
OPTIMIZING NDOUTILS TABLE: nagios_timedeventqueue
SQL: OPTIMIZE TABLE nagios_timedeventqueue
OPTIMIZING NDOUTILS TABLE: nagios_timedevents
SQL: OPTIMIZE TABLE nagios_timedevents
OPTIMIZING NDOUTILS TABLE: nagios_timeperiod_timeranges
SQL: OPTIMIZE TABLE nagios_timeperiod_timeranges
OPTIMIZING NDOUTILS TABLE: nagios_timeperiods
SQL: OPTIMIZE TABLE nagios_timeperiods
CLEANING nagiosxi TABLE 'commands'...
SQL: DELETE FROM xi_commands WHERE processing_time < 1518193501::abstime::timestamp without time zone
CLEANING nagiosxi TABLE 'events'...
SQL: DELETE FROM xi_events WHERE processing_time < 1518193501::abstime::timestamp without time zone
SQL1: SELECT xi_meta.meta_id FROM xi_meta LEFT JOIN xi_events ON xi_meta.metaobj_id=xi_events.event_id WHERE metatype_id='1' AND event_id IS NULL
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' EXISTS - EXITING!
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' EXISTS - EXITING!
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' EXISTS - EXITING!
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' EXISTS - EXITING!
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' EXISTS - EXITING!
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' EXISTS - EXITING!
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' IS OLD - REMOVING
CREATING: /usr/local/nagiosxi/var/dbmaint.lock
CLEANING ndoutils TABLE 'commenthistory'...
SQL: DELETE FROM nagios_commenthistory WHERE entry_time < FROM_UNIXTIME(1486688401)
CLEANING ndoutils TABLE 'processevents'...
SQL: DELETE FROM nagios_processevents WHERE event_time < FROM_UNIXTIME(1486688401)
CLEANING ndoutils TABLE 'externalcommands'...
SQL: DELETE FROM nagios_externalcommands WHERE entry_time < FROM_UNIXTIME(1517619601)
CLEANING ndoutils TABLE 'logentries'...
SQL: DELETE FROM nagios_logentries WHERE logentry_time < FROM_UNIXTIME(1510448401)
CLEANING ndoutils TABLE 'notifications'...
SQL: DELETE FROM nagios_notifications WHERE start_time < FROM_UNIXTIME(1510448401)
CLEANING ndoutils TABLE 'contactnotifications'...
SQL: DELETE FROM nagios_contactnotifications WHERE start_time < FROM_UNIXTIME(1510448401)
CLEANING ndoutils TABLE 'contactnotificationmethods'...
SQL: DELETE FROM nagios_contactnotificationmethods WHERE start_time < FROM_UNIXTIME(1510448401)
CLEANING ndoutils TABLE 'statehistory'...
SQL: DELETE FROM nagios_statehistory WHERE state_time < FROM_UNIXTIME(1455152401)
CLEANING ndoutils TABLE 'timedevents'...
SQL: DELETE FROM nagios_timedevents WHERE event_time < FROM_UNIXTIME(1518224101)
CLEANING ndoutils TABLE 'systemcommands'...
SQL: DELETE FROM nagios_systemcommands WHERE start_time < FROM_UNIXTIME(1518224101)
CLEANING ndoutils TABLE 'servicechecks'...
SQL: DELETE FROM nagios_servicechecks WHERE start_time < FROM_UNIXTIME(1518224101)
CLEANING ndoutils TABLE 'hostchecks'...
SQL: DELETE FROM nagios_hostchecks WHERE start_time < FROM_UNIXTIME(1518224101)
CLEANING ndoutils TABLE 'eventhandlers'...
SQL: DELETE FROM nagios_eventhandlers WHERE start_time < FROM_UNIXTIME(1518224101)
LASTOPT: 1518222301
INTERVAL: 60
NOW: 1518224401
OPTTIME: 1518225901
CLEANING nagiosxi TABLE 'commands'...
SQL: DELETE FROM xi_commands WHERE processing_time < 1518195601::abstime::timestamp without time zone
CLEANING nagiosxi TABLE 'events'...
SQL: DELETE FROM xi_events WHERE processing_time < 1518195601::abstime::timestamp without time zone
SQL1: SELECT xi_meta.meta_id FROM xi_meta LEFT JOIN xi_events ON xi_meta.metaobj_id=xi_events.event_id WHERE metatype_id='1' AND event_id IS NULL
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' EXISTS - EXITING!
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' EXISTS - EXITING!
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' EXISTS - EXITING!
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' EXISTS - EXITING!
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' EXISTS - EXITING!
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' EXISTS - EXITING!
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' IS OLD - REMOVING
CREATING: /usr/local/nagiosxi/var/dbmaint.lock
CLEANING ndoutils TABLE 'commenthistory'...
SQL: DELETE FROM nagios_commenthistory WHERE entry_time < FROM_UNIXTIME(1486690501)
CLEANING ndoutils TABLE 'processevents'...
SQL: DELETE FROM nagios_processevents WHERE event_time < FROM_UNIXTIME(1486690501)
CLEANING ndoutils TABLE 'externalcommands'...
SQL: DELETE FROM nagios_externalcommands WHERE entry_time < FROM_UNIXTIME(1517621701)
CLEANING ndoutils TABLE 'logentries'...
SQL: DELETE FROM nagios_logentries WHERE logentry_time < FROM_UNIXTIME(1510450501)
CLEANING ndoutils TABLE 'notifications'...
SQL: DELETE FROM nagios_notifications WHERE start_time < FROM_UNIXTIME(1510450501)
CLEANING ndoutils TABLE 'contactnotifications'...
SQL: DELETE FROM nagios_contactnotifications WHERE start_time < FROM_UNIXTIME(1510450501)
CLEANING ndoutils TABLE 'contactnotificationmethods'...
SQL: DELETE FROM nagios_contactnotificationmethods WHERE start_time < FROM_UNIXTIME(1510450501)
CLEANING ndoutils TABLE 'statehistory'...
SQL: DELETE FROM nagios_statehistory WHERE state_time < FROM_UNIXTIME(1455154501)
CLEANING ndoutils TABLE 'timedevents'...
SQL: DELETE FROM nagios_timedevents WHERE event_time < FROM_UNIXTIME(1518226201)
CLEANING ndoutils TABLE 'systemcommands'...
SQL: DELETE FROM nagios_systemcommands WHERE start_time < FROM_UNIXTIME(1518226201)
CLEANING ndoutils TABLE 'servicechecks'...
SQL: DELETE FROM nagios_servicechecks WHERE start_time < FROM_UNIXTIME(1518226201)
CLEANING ndoutils TABLE 'hostchecks'...
SQL: DELETE FROM nagios_hostchecks WHERE start_time < FROM_UNIXTIME(1518226201)
CLEANING ndoutils TABLE 'eventhandlers'...
SQL: DELETE FROM nagios_eventhandlers WHERE start_time < FROM_UNIXTIME(1518226201)
TIME TO OPTIMIZE
LASTOPT: 1518222301
INTERVAL: 60
NOW: 1518226501
OPTTIME: 1518225901
OPTIMIZING NDOUTILS TABLE: nagios_acknowledgements
SQL: OPTIMIZE TABLE nagios_acknowledgements
OPTIMIZING NDOUTILS TABLE: nagios_commands
SQL: OPTIMIZE TABLE nagios_commands
OPTIMIZING NDOUTILS TABLE: nagios_commenthistory
SQL: OPTIMIZE TABLE nagios_commenthistory
OPTIMIZING NDOUTILS TABLE: nagios_comments
SQL: OPTIMIZE TABLE nagios_comments
OPTIMIZING NDOUTILS TABLE: nagios_configfiles
SQL: OPTIMIZE TABLE nagios_configfiles
OPTIMIZING NDOUTILS TABLE: nagios_configfilevariables
SQL: OPTIMIZE TABLE nagios_configfilevariables
OPTIMIZING NDOUTILS TABLE: nagios_conninfo
SQL: OPTIMIZE TABLE nagios_conninfo
OPTIMIZING NDOUTILS TABLE: nagios_contact_addresses
SQL: OPTIMIZE TABLE nagios_contact_addresses
OPTIMIZING NDOUTILS TABLE: nagios_contact_notificationcommands
SQL: OPTIMIZE TABLE nagios_contact_notificationcommands
OPTIMIZING NDOUTILS TABLE: nagios_contactgroup_members
SQL: OPTIMIZE TABLE nagios_contactgroup_members
OPTIMIZING NDOUTILS TABLE: nagios_contactgroups
SQL: OPTIMIZE TABLE nagios_contactgroups
OPTIMIZING NDOUTILS TABLE: nagios_contactnotificationmethods
SQL: OPTIMIZE TABLE nagios_contactnotificationmethods
OPTIMIZING NDOUTILS TABLE: nagios_contactnotifications
SQL: OPTIMIZE TABLE nagios_contactnotifications
OPTIMIZING NDOUTILS TABLE: nagios_contacts
SQL: OPTIMIZE TABLE nagios_contacts
OPTIMIZING NDOUTILS TABLE: nagios_contactstatus
SQL: OPTIMIZE TABLE nagios_contactstatus
OPTIMIZING NDOUTILS TABLE: nagios_customvariables
SQL: OPTIMIZE TABLE nagios_customvariables
OPTIMIZING NDOUTILS TABLE: nagios_customvariablestatus
SQL: OPTIMIZE TABLE nagios_customvariablestatus
OPTIMIZING NDOUTILS TABLE: nagios_dbversion
SQL: OPTIMIZE TABLE nagios_dbversion
OPTIMIZING NDOUTILS TABLE: nagios_downtimehistory
SQL: OPTIMIZE TABLE nagios_downtimehistory
OPTIMIZING NDOUTILS TABLE: nagios_eventhandlers
SQL: OPTIMIZE TABLE nagios_eventhandlers
OPTIMIZING NDOUTILS TABLE: nagios_externalcommands
SQL: OPTIMIZE TABLE nagios_externalcommands
OPTIMIZING NDOUTILS TABLE: nagios_flappinghistory
SQL: OPTIMIZE TABLE nagios_flappinghistory
OPTIMIZING NDOUTILS TABLE: nagios_host_contactgroups
SQL: OPTIMIZE TABLE nagios_host_contactgroups
OPTIMIZING NDOUTILS TABLE: nagios_host_contacts
SQL: OPTIMIZE TABLE nagios_host_contacts
OPTIMIZING NDOUTILS TABLE: nagios_host_parenthosts
SQL: OPTIMIZE TABLE nagios_host_parenthosts
OPTIMIZING NDOUTILS TABLE: nagios_hostchecks
SQL: OPTIMIZE TABLE nagios_hostchecks
OPTIMIZING NDOUTILS TABLE: nagios_hostdependencies
SQL: OPTIMIZE TABLE nagios_hostdependencies
OPTIMIZING NDOUTILS TABLE: nagios_hostescalation_contactgroups
SQL: OPTIMIZE TABLE nagios_hostescalation_contactgroups
OPTIMIZING NDOUTILS TABLE: nagios_hostescalation_contacts
SQL: OPTIMIZE TABLE nagios_hostescalation_contacts
OPTIMIZING NDOUTILS TABLE: nagios_hostescalations
SQL: OPTIMIZE TABLE nagios_hostescalations
OPTIMIZING NDOUTILS TABLE: nagios_hostgroup_members
SQL: OPTIMIZE TABLE nagios_hostgroup_members
OPTIMIZING NDOUTILS TABLE: nagios_hostgroups
SQL: OPTIMIZE TABLE nagios_hostgroups
OPTIMIZING NDOUTILS TABLE: nagios_hosts
SQL: OPTIMIZE TABLE nagios_hosts
OPTIMIZING NDOUTILS TABLE: nagios_hoststatus
SQL: OPTIMIZE TABLE nagios_hoststatus
OPTIMIZING NDOUTILS TABLE: nagios_instances
SQL: OPTIMIZE TABLE nagios_instances
OPTIMIZING NDOUTILS TABLE: nagios_logentries
SQL: OPTIMIZE TABLE nagios_logentries
OPTIMIZING NDOUTILS TABLE: nagios_notifications
SQL: OPTIMIZE TABLE nagios_notifications
OPTIMIZING NDOUTILS TABLE: nagios_objects
SQL: OPTIMIZE TABLE nagios_objects
OPTIMIZING NDOUTILS TABLE: nagios_processevents
SQL: OPTIMIZE TABLE nagios_processevents
OPTIMIZING NDOUTILS TABLE: nagios_programstatus
SQL: OPTIMIZE TABLE nagios_programstatus
OPTIMIZING NDOUTILS TABLE: nagios_runtimevariables
SQL: OPTIMIZE TABLE nagios_runtimevariables
OPTIMIZING NDOUTILS TABLE: nagios_scheduleddowntime
SQL: OPTIMIZE TABLE nagios_scheduleddowntime
OPTIMIZING NDOUTILS TABLE: nagios_service_contactgroups
SQL: OPTIMIZE TABLE nagios_service_contactgroups
OPTIMIZING NDOUTILS TABLE: nagios_service_contacts
SQL: OPTIMIZE TABLE nagios_service_contacts
OPTIMIZING NDOUTILS TABLE: nagios_servicechecks
SQL: OPTIMIZE TABLE nagios_servicechecks
OPTIMIZING NDOUTILS TABLE: nagios_servicedependencies
SQL: OPTIMIZE TABLE nagios_servicedependencies
OPTIMIZING NDOUTILS TABLE: nagios_serviceescalation_contactgroups
SQL: OPTIMIZE TABLE nagios_serviceescalation_contactgroups
OPTIMIZING NDOUTILS TABLE: nagios_serviceescalation_contacts
SQL: OPTIMIZE TABLE nagios_serviceescalation_contacts
OPTIMIZING NDOUTILS TABLE: nagios_serviceescalations
SQL: OPTIMIZE TABLE nagios_serviceescalations
OPTIMIZING NDOUTILS TABLE: nagios_servicegroup_members
SQL: OPTIMIZE TABLE nagios_servicegroup_members
OPTIMIZING NDOUTILS TABLE: nagios_servicegroups
SQL: OPTIMIZE TABLE nagios_servicegroups
OPTIMIZING NDOUTILS TABLE: nagios_services
SQL: OPTIMIZE TABLE nagios_services
OPTIMIZING NDOUTILS TABLE: nagios_servicestatus
SQL: OPTIMIZE TABLE nagios_servicestatus
OPTIMIZING NDOUTILS TABLE: nagios_statehistory
SQL: OPTIMIZE TABLE nagios_statehistory
OPTIMIZING NDOUTILS TABLE: nagios_systemcommands
SQL: OPTIMIZE TABLE nagios_systemcommands
OPTIMIZING NDOUTILS TABLE: nagios_timedeventqueue
SQL: OPTIMIZE TABLE nagios_timedeventqueue
OPTIMIZING NDOUTILS TABLE: nagios_timedevents
SQL: OPTIMIZE TABLE nagios_timedevents
OPTIMIZING NDOUTILS TABLE: nagios_timeperiod_timeranges
SQL: OPTIMIZE TABLE nagios_timeperiod_timeranges
OPTIMIZING NDOUTILS TABLE: nagios_timeperiods
SQL: OPTIMIZE TABLE nagios_timeperiods
CLEANING nagiosxi TABLE 'commands'...
SQL: DELETE FROM xi_commands WHERE processing_time < 1518197701::abstime::timestamp without time zone
CLEANING nagiosxi TABLE 'events'...
SQL: DELETE FROM xi_events WHERE processing_time < 1518197701::abstime::timestamp without time zone
SQL1: SELECT xi_meta.meta_id FROM xi_meta LEFT JOIN xi_events ON xi_meta.metaobj_id=xi_events.event_id WHERE metatype_id='1' AND event_id IS NULL
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' EXISTS - EXITING!
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' EXISTS - EXITING!
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' EXISTS - EXITING!
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' EXISTS - EXITING!
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' EXISTS - EXITING!
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' EXISTS - EXITING!
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' IS OLD - REMOVING
CREATING: /usr/local/nagiosxi/var/dbmaint.lock
CLEANING ndoutils TABLE 'commenthistory'...
SQL: DELETE FROM nagios_commenthistory WHERE entry_time < FROM_UNIXTIME(1486692601)
CLEANING ndoutils TABLE 'processevents'...
SQL: DELETE FROM nagios_processevents WHERE event_time < FROM_UNIXTIME(1486692601)
CLEANING ndoutils TABLE 'externalcommands'...
SQL: DELETE FROM nagios_externalcommands WHERE entry_time < FROM_UNIXTIME(1517623801)
CLEANING ndoutils TABLE 'logentries'...
SQL: DELETE FROM nagios_logentries WHERE logentry_time < FROM_UNIXTIME(1510452601)
CLEANING ndoutils TABLE 'notifications'...
SQL: DELETE FROM nagios_notifications WHERE start_time < FROM_UNIXTIME(1510452601)
CLEANING ndoutils TABLE 'contactnotifications'...
SQL: DELETE FROM nagios_contactnotifications WHERE start_time < FROM_UNIXTIME(1510452601)
CLEANING ndoutils TABLE 'contactnotificationmethods'...
SQL: DELETE FROM nagios_contactnotificationmethods WHERE start_time < FROM_UNIXTIME(1510452601)
CLEANING ndoutils TABLE 'statehistory'...
SQL: DELETE FROM nagios_statehistory WHERE state_time < FROM_UNIXTIME(1455156601)
CLEANING ndoutils TABLE 'timedevents'...
SQL: DELETE FROM nagios_timedevents WHERE event_time < FROM_UNIXTIME(1518228301)
CLEANING ndoutils TABLE 'systemcommands'...
SQL: DELETE FROM nagios_systemcommands WHERE start_time < FROM_UNIXTIME(1518228301)
CLEANING ndoutils TABLE 'servicechecks'...
SQL: DELETE FROM nagios_servicechecks WHERE start_time < FROM_UNIXTIME(1518228301)
CLEANING ndoutils TABLE 'hostchecks'...
SQL: DELETE FROM nagios_hostchecks WHERE start_time < FROM_UNIXTIME(1518228301)
CLEANING ndoutils TABLE 'eventhandlers'...
SQL: DELETE FROM nagios_eventhandlers WHERE start_time < FROM_UNIXTIME(1518228301)
LASTOPT: 1518226501
INTERVAL: 60
NOW: 1518228601
OPTTIME: 1518230101
CLEANING nagiosxi TABLE 'commands'...
SQL: DELETE FROM xi_commands WHERE processing_time < 1518199801::abstime::timestamp without time zone
CLEANING nagiosxi TABLE 'events'...
SQL: DELETE FROM xi_events WHERE processing_time < 1518199801::abstime::timestamp without time zone
SQL1: SELECT xi_meta.meta_id FROM xi_meta LEFT JOIN xi_events ON xi_meta.metaobj_id=xi_events.event_id WHERE metatype_id='1' AND event_id IS NULL
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' EXISTS - EXITING!
LOCKFILE '/usr/local/nagiosxi/var/dbmaint.lock' EXISTS - EXITING!
Re: Nagios not sending out notifications
Posted: Sat Feb 10, 2018 6:54 am
by jkinning
Like the Energizer battery... still going.
Code: Select all
***** Nagios XI Alert *****
Nagios has detected a problem with this service.
Notification Type: PROBLEM
Service: Nagios XI Jobs
Host: nagiosl1p
Address:
State: CRITICAL
Info:
Database Maintenance (dbmaint) stale (95191 seconds old), Database Maintenance (dbmaint) stale (95191 seconds old)
Date/Time: 2018-02-10 06:33:47
Re: Nagios not sending out notifications
Posted: Mon Feb 12, 2018 12:07 pm
by npolovenko
@jkinning, It's interesting because you deleted the lock file, but the message still says the file is more than 17 hours stale.
Can you run:
Code: Select all
ll /usr/local/nagiosxi/var/dbmaint.lock
This will show us the age of the file, and we could see if it was really removed when you ran the commands that I gave you.
Let's try a more advanced way of removing dmaint.sock, this hopefully will actually get it removed.
Code: Select all
service crond stop
service nagios stop
service mysqld stop
service mariadb stop
rm /usr/local/nagiosxi/var/dbmaint.lock
check to see if there's a dmaint.sock or a similar file in the same directory and if there remove it as well!
service crond start
service nagios start
service mysqld start
service mariadb start
Re: Nagios not sending out notifications
Posted: Mon Feb 12, 2018 12:14 pm
by lmiltchev
By looking at your notification, it seems like that you created this service by running the "Nagios XI Server" monitoring wizard. How many XI servers do you have? Can you run the following command on your "main" XI box, and show the output?
Code: Select all
/usr/bin/php /usr/local/nagios/libexec/check_nagiosxiserver.php --address=<nagiosl1p ip address> --url=<url> --username=nagiosadmin --ticket=<"xxx"> --mode=jobs --debug=1
Substitute the text in the "<>" with the actual values. I hope that we can find some clues in the debug output. Also, can you run the following command on the 'nagiosl1p' box, and show the output?
Code: Select all
/usr/bin/php -q /usr/local/nagiosxi/cron/dbmaint.php
After this, go to your main XI server, find the "Nagios XI Jobs" check under the "Server Detail" page, and schedule a forced immediate check. Did the output change?
Re: Nagios not sending out notifications
Posted: Tue Feb 13, 2018 8:30 am
by jkinning
After several hours >8 I finally got the recovery notification. Not sure what the issue was but things are looking green and operational again.
Re: Nagios not sending out notifications
Posted: Tue Feb 13, 2018 10:36 am
by lmiltchev
Great! I am glad your issue has been resolved! I will be locking this topic now. If you have any more issues/questions, please start a new thread.