How to monitor local services?

[vmesquita]

Hello,

In Troy Lea's talk "Nagios XI Best Practices", at Nagios Conference 2015, he suggests that it's advisable to monitor some local services like crond, httpd, ndo2db etc. However I can't figure out how to do it (except for writing a custom plugin, but I imagine there's a more straighfoward way). Check_init_service, as he suggests on the video, doesn't exist as an standalone plugin, just as a part of check_nrpe. How can I make this work?

[WillemDH]

You should have a command check_local_init_service, which is made to monitor local services. Use that.

[vmesquita]

Aparently not... I just went /usr/local and did:

Code: Select all

find * | grep check_local_init_service

with no results... Or is it supposed to be in another folder? I am using Nagios XI 5.2.

[WillemDH]

No sorry it is a command. You can find it in the CCM => Commands. Configure your service with it.

[vmesquita]

I just checked and there's no check_serviecs_command defined on XI interface. Also there's no file check_init_service in /usr/local/nagios/libexec. Is it supposed to come with the regular install?

[WillemDH]

If it doesn't exist, it's easy to create a new command. You can check the needed config in my screenshot.

[vmesquita]

But the check_init_service script does not exist in /usr/local/nagios/libexec... So it makes no sense to create a script pointing to a non-existant file. Maybe my install is missing this file for some reason. Could you post this script?

[WillemDH]

This is it:

Code: Select all

#!/usr/bin/env python                                                                                             
                                                                                                                  
# check_init_service ; -*-Python-*-                                                                               
# a simple nagios check to verify if init scripts are running                                                     
# Copyright James Powell 2013 / jamespo [at] gmail [dot] com                                                      
# This program is distributed under the terms of the GNU General Public License v3                                
                                                                                                                  
import sys, re, os                                                                                                
from optparse import OptionParser                                                                                 
                                                                                                                  
class CheckInitService(object):                                                                                   
    def __init__(self, options):                                                                                  
        self.services = options.services.split(',')                                                               
        self.matchregex = options.matchregex                                                                      
        self.svccmd = options.svccmd                                                                              
                                                                                                                  
    @staticmethod                                                                                                 
    def _findservice():                                                                                           
        '''return full path to service command'''                                                                 
        for svc in ['/usr/sbin/service', '/sbin/service']:                                                        
            if os.path.isfile(svc):                                                                               
                return svc                                                                                        
        return None                                                                                               
                                                                                                                  
    @staticmethod                                                                                                 
    def build_cmdline(svc_cmd, servicename):                                                                      
        if svc_cmd == "/bin/systemctl":                                                                           
            return "%s is-active %s" % (svc_cmd, servicename)                                                     
        else:                                                                                                     
            return '/usr/bin/sudo -n ' + svc_cmd + ' ' + servicename + ' status 2>&1'                             
                                                                                                                  
    def checkinits(self):                                                                                         
        '''check init scripts statuses'''                                                                         
        self.running_services = dict()                                                                            
        self.failing_services = dict()                                                                            
        if self.svccmd is None:                                                                                   
            svc_cmd = self._findservice()                                                                         
        else:                                                                                                     
            svc_cmd = self.svccmd                                                                                 
        # loop round all the services                                                                             
        for servicename in self.services:                                                                         
            cmdline = self.build_cmdline(svc_cmd, servicename)                                                    
            initresults = [line.strip() for line in os.popen(cmdline).readlines()]                                
            for res in initresults:                                                                               
                if re.search(self.matchregex, res) is not None:    
                    self.running_services[servicename] = res
                    break
                else:
                    self.failing_services[servicename] = res
        # if # of ok results == # of services checked, all ok
        if len(self.running_services) == len(self.services):
            return 0
        else:
            return 1

def main():
    parser = OptionParser()
    parser.add_option("--services", dest="services", help="service1,service2")
    parser.add_option("--matchregex", dest="matchregex", default="(?:^active|is running|start/running)",
        help="regex to match running service status")
    parser.add_option("--svccmd", dest="svccmd", help="full path to command to run for check")
    (options, args) = parser.parse_args()
    if options.services is None:
        print "UNKNOWN: No services specified"
        sys.exit(2)
    ci = CheckInitService(options)
    rc = ci.checkinits()
    if rc == 0:
        rcstr = 'OK: all services running (' + ','.join(ci.running_services) + ')'
    else:
        rcstr = 'CRITICAL: Failing (' +  ','.join(ci.failing_services) + ')'
        if len(ci.running_services) > 0:
            rcstr += ' Running (' + '.'.join(ci.running_services) + ')'
    print rcstr
    sys.exit(rc)

if __name__ == '__main__':
    main()

[vmesquita]

Thanks! It worked.

[rkennedy]

Thanks @WillemDH!

Glad to see you have it working. I'll close this thread out now, but feel free to open another thread if you ever need more assistance.