Taking server out of service while it's already marked DOWN

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
gossamer
Posts: 20
Joined: Wed Jul 13, 2011 9:56 pm

Taking server out of service while it's already marked DOWN

Post by gossamer »

Hi,

I have a nagios-2.0.1 system on fedora21 and have a server down at the colo that I won't be able to attend to until at least three days from now. It's constantly alerting me that it's down, but nothing I seem to be able to do will tell nagios to stop monitoring it until I can fix it.

Is there a way to take it out of service until it's back online after it's already down?
ssax
Dreams In Code
Posts: 7682
Joined: Wed Feb 11, 2015 12:54 pm

Re: Taking server out of service while it's already marked D

Post by ssax »

Have you tried scheduling downtime for the host or have you clicked the disable notifications on the host? Are you getting any errors when doing that?

If you edit the config file of the host and services you should be able to specify notifications_enabled 0 and restart the nagios service.

Worst case scenario you could set notifications_enabled=0 in your /usr/local/nagios/etc/nagios.cfg file and restart the nagios service to stop notifications.
gossamer
Posts: 20
Joined: Wed Jul 13, 2011 9:56 pm

Re: Taking server out of service while it's already marked D

Post by gossamer »

ssax wrote:Have you tried scheduling downtime for the host or have you clicked the disable notifications on the host? Are you getting any errors when doing that?

If you edit the config file of the host and services you should be able to specify notifications_enabled 0 and restart the nagios service.

Worst case scenario you could set notifications_enabled=0 in your /usr/local/nagios/etc/nagios.cfg file and restart the nagios service to stop notifications.
Maybe I haven't set it up right. I have a single services.cfg which defines all the services for all the hosts being monitored. At the top of the file, I have a "define service" with a "name standard-service-24x7" which is the only "define service" that has a name. It is here where notifications_enabled is set. Then each service for each host looks similar to this:

define service {
use standard-service-24x7
host_name pixie
service_description NTP
check_command check_nrpe!check_ntp
}

repeating for each host and each service on each host. So, I would only be able to disable all notifications to all hosts if I were to set notifications_enabled to 0.

I suppose I could add that to each service, but that's a lot of work...
jdalrymple
Skynet Drone
Posts: 2620
Joined: Wed Feb 11, 2015 1:56 pm

Re: Taking server out of service while it's already marked D

Post by jdalrymple »

You can apply the "notifications_enabled" directive in the very lowest layer of your service definition and it will only apply to the service/host combination there. It will not affect stuff in templates upstream. If the definition you showed applied that service to multiple hosts it could be an issue, that doesn't appear to be the case though.

Actually you have stuff set up quite nicely, not wrong at all.

Does what I described make sense?
gossamer
Posts: 20
Joined: Wed Jul 13, 2011 9:56 pm

Re: Taking server out of service while it's already marked D

Post by gossamer »

jdalrymple wrote:You can apply the "notifications_enabled" directive in the very lowest layer of your service definition and it will only apply to the service/host combination there. It will not affect stuff in templates upstream. If the definition you showed applied that service to multiple hosts it could be an issue, that doesn't appear to be the case though.

Actually you have stuff set up quite nicely, not wrong at all.

Does what I described make sense?
Yes, I believe it makes sense, but changing it for every service is quite laborious. Is there no shorter/faster way to do it?

Is this the equivalent of setting "Disable notifications for all services on this host" from within the web front-end?
jdalrymple
Skynet Drone
Posts: 2620
Joined: Wed Feb 11, 2015 1:56 pm

Re: Taking server out of service while it's already marked D

Post by jdalrymple »

gossamer wrote:and have a server down at the colo that I won't be able to attend to until at least three days from now
Shouldn't take long for "a server"?

Also, the generally accepted way of dealing with this is to just disable notifications in the CGI or put the server into a downtime. Why not use one of those options (suggested above by ssax)?
Attachments
Clipboard01.jpg
gossamer
Posts: 20
Joined: Wed Jul 13, 2011 9:56 pm

Re: Taking server out of service while it's already marked D

Post by gossamer »

jdalrymple wrote:
gossamer wrote:and have a server down at the colo that I won't be able to attend to until at least three days from now
Shouldn't take long for "a server"?

Also, the generally accepted way of dealing with this is to just disable notifications in the CGI or put the server into a downtime. Why not use one of those options (suggested above by ssax)?
Thanks for your help, guys. I recall trying to use the "Schedule Downtime" option after the server was already down, and it continuing to alert me the server was down. I guess I'll just have to play with the combination of options until I find the ones that work.
User avatar
tgriep
Madmin
Posts: 9190
Joined: Thu Oct 30, 2014 9:02 am

Re: Taking server out of service while it's already marked D

Post by tgriep »

Were you receiving Host notifications or Service Notifications, or both?
When you schedule downtime, you will have to schedule downtime for host and services separately.
Be sure to check out our Knowledgebase for helpful articles and solutions!
gossamer
Posts: 20
Joined: Wed Jul 13, 2011 9:56 pm

Re: Taking server out of service while it's already marked D

Post by gossamer »

tgriep wrote:Were you receiving Host notifications or Service Notifications, or both?
When you schedule downtime, you will have to schedule downtime for host and services separately.
Good to know. Thanks so much for the info.
User avatar
hsmith
Agent Smith
Posts: 3539
Joined: Thu Jul 30, 2015 11:09 am
Location: 127.0.0.1
Contact:

Re: Taking server out of service while it's already marked D

Post by hsmith »

gossamer wrote:
tgriep wrote:Were you receiving Host notifications or Service Notifications, or both?
When you schedule downtime, you will have to schedule downtime for host and services separately.
Good to know. Thanks so much for the info.
Is there anything else we can do to help you, or is this one all right to close?
Former Nagios Employee.
me.
Locked