Notifications if Remote Nagios Service is down

[gr8matt]

All,

I have a number of Nagios machines running on customer sites around my city. I have OpenVPN tunnels to each of these so I can remotely login and monitor health, do remote break/fix, etc.

Recently, I logged into one of my customers sites and found that Nagios had shutdown and was not running (most likely a missed init.d config). I would now like to have my local Nagios system to notify me if a remote nagios service is shut down. What is the best way to do this? Should I run NRPE and Nagios on the same machine and query using NRPE from my local machine to the remote one?

Any thoughts?

Matt

[hsmith]

I really think that would be an effective way of doing it. For instance, from my XI machine to my Core machine:

Code: Select all

[nagios@xi libexec]$ ./check_nrpe -H 172.16.0.19 -c check_init_service -a nagios
nagios (pid 56110) is running...
[nagios@xi libexec]$

[eloyd]

Total agreement. If you assume your Nagios servers are arranged in a circle, have each one check it's clockwise and/or anti-clockwise neighbor to see if the nagios software is running. Alert as appropriate. NRPE over your private VPN IPs is the best way to do this.

You could also do it with passive checks to have each Nagios box tell another Nagios box that it's still there, and then alert on freshness checks failing (https://assets.nagios.com/downloads/nag ... hness.html, use a "check_dummy 2" check to force the stale check to indicate a critical result, for instance). Now you can treat a "no updates in X minutes" scenario as a critical and take alerting actions. No NRPE required.

[hsmith]

Thanks Eric.

[gr8matt]

Thanks all! That worked perfectly. I got NRPE running on all the systems across the board this morning. I did run into some problems with check_init_services plugin (I am sure I was doing something wrong) but settled on "check_procs -C nagios. It normally has 6-8 processes running so I have it warn if it is 3 and go critical if it is 1 or under... I tested it on a few systems and it works flawlessly!

Thanks again! You guys rock!

Matt

[hsmith]

Glad to hear that it is working. Are we all right to close this one?

[eloyd]

Likely your check_init_services check needs to run as root, which means you need to set nagios up to be able to use sudo and then update the nrpe.cfg file on the remote host to use sudo to execute the command. Your work-around is perfectly fine, though.

Glad we could help!

[tmcdonald]

I'll be closing this thread now, but feel free to open another if you need anything in the future!