Monitoring Jenkins slaves (PowerShell)

[Beandip408]

Does anyone here have any experience of monitoring Jenkins slaves using Nagios Core?

Some of my build servers can have a Jenkins slave on more than one Jenkins master, and i want to be able to monitor them individually, rather than just monitor java.exe.

Here is the original thread that i was following: https://support.nagios.com/forum/viewto ... 45#p176645

i followed these steps:

Code: Select all

In your nsclient.ini add something like this under [/settings/external scripts/scripts]

CODE: SELECT ALL
check_pstate = cmd /c echo scripts\check_pstate.ps1 "$ARG1$"; exit($lastexitcode) | powershell.exe -command -


Create the powershell script in your NSClient++\scripts directory.

Your powershell script would need to check for the command line details like you've been trying, you could use something like this:

CODE: SELECT ALL
Get-WmiObject Win32_Process -Filter "CommandLine like '%your unique commandline details%'"


if it finds it exit 0, if it doesn't, exit 2

Because im still learning Nagios, im not sure how to implement this from my nagios server. I could use some help getting the check_command info implemented on the server side to check for 3 of these running processes.

Thanks

[ssax]

Did you already write the powershell script to perform the check?

Are you already checking these servers with NRPE at all?

You can go to Configure > Configuration Wizards > NRPE
- Display Name: Whatever you'd like
- Remote NRPE Command: check_pstate
- Command Args: 'your unique commandline details'

Then continue the wizard and let it apply config.

That should get you up and running.

Let us know the results.

[Beandip408]

Did you already write the powershell script to perform the check?

Are you already checking these servers with NRPE at all?

You can go to Configure > Configuration Wizards > NRPE
- Display Name: Whatever you'd like
- Remote NRPE Command: check_pstate
- Command Args: 'your unique commandline details'

Then continue the wizard and let it apply config.

That should get you up and running.

Let us know the results.

i wrote the powershell script with my arg inline:

Code: Select all

Get-WmiObject Win32_Process -Filter "CommandLine like '%com.domain.reportgenerator.ReportGenerator%'"

on the windows host i edited the nsclient.ini:

Code: Select all

check_rgstate = cmd /c echo scripts\check_rgstate.ps1 "$ARG1$"; exit($lastexitcode) | powershell.exe -command -

and then on my nagios server (linux):

Code: Select all

define service {
        use                             critical-service
        hostgroup_name                  test_rptgen
        service_description             Process - ReportGen
        check_command                   check_nrpe!check_rgstate!warn=2!crit=1
        notifications_enabled   0
}

But running the check from the nagios server returns this:

Code: Select all

[root@nagiosServer services]# /lib/nagios/libexec/check_nrpe -H rptgen01 -c check_rgstate


__GENUS                    : 2
__CLASS                    : Win32_Process
__SUPERCLASS               : CIM_Process
__DYNASTY                  : CIM_ManagedSystemElement
__RELPATH                  : Win32_Process.Handle="5820"
__PROPERTY_COUNT           : 45
__DERIVATION               : {CIM_Process, CIM_LogicalElement,
                             CIM_ManagedSystemElement}
__SERVER                   : RPTGEN01
__NAMESPACE                : root\cimv2
__PATH                     : \\RPTGEN01\root\cimv2:Win32_Process.Handle="5
                             820"
Caption                    : java.exe
CommandLine                : java.exe  -Xmx4096m
                             com.domain.reportgenerator.ReportGenerator
CreationClassName          : Win32_Process
CreationDate               : 20160616153524.495967-360
CSCreationClassName        : Win32_ComputerSystem
CSName                     : RPTGEN01
Description                : java.exe
ExecutablePath             : C:\Pro

I even tried editing the powershell script to count and still no go.:

Code: Select all

(Get-WmiObject Win32_Process -Filter "CommandLine like '%com.domain.reportgenerator.ReportGenerator%'").count

any ideas what im doing wrong?

[ssax]

Your plugin needs to output data (and perfdata if you want it) and an exit code for it to work properly:

https://assets.nagios.com/downloads/nag ... inapi.html

So, you would modify the powershell script and have it do something like this:

Code: Select all

pCount = (Get-WmiObject Win32_Process -Filter "CommandLine like '%com.domain.reportgenerator.ReportGenerator%'").count

if (pCount -gt 0) {
	write-host "OK - Found 1 proces"
	exit 0
} elseif (pCount -lt 1) {
	write-host "CRITICAL - No process found"
	exit 2
}

[Beandip408]

Your plugin needs to output data (and perfdata if you want it) and an exit code for it to work properly:

https://assets.nagios.com/downloads/nag ... inapi.html

So, you would modify the powershell script and have it do something like this:

Code: Select all

pCount = (Get-WmiObject Win32_Process -Filter "CommandLine like '%com.domain.reportgenerator.ReportGenerator%'").count

if (pCount -gt 0) {
	write-host "OK - Found 1 proces"
	exit 0
} elseif (pCount -lt 1) {
	write-host "CRITICAL - No process found"
	exit 2
}

Thank you. After implementing this script, im still seeing this on the nagios server:

Code: Select all

[root@nagiosServer services]# /lib/nagios/libexec/check_nrpe -H rptgen01 -c check_rgstate


__GENUS                    : 2
__CLASS                    : Win32_Process
__SUPERCLASS               : CIM_Process
__DYNASTY                  : CIM_ManagedSystemElement
__RELPATH                  : Win32_Process.Handle="5820"
__PROPERTY_COUNT           : 45
__DERIVATION               : {CIM_Process, CIM_LogicalElement,
                             CIM_ManagedSystemElement}
__SERVER                   : RPTGEN01
__NAMESPACE                : root\cimv2
__PATH                     : \\RPTGEN01\root\cimv2:Win32_Process.Handle="5
                             820"
Caption                    : java.exe
CommandLine                : java.exe  -Xmx4096m
                             com.domain.reportgenerator.ReportGenerator
CreationClassName          : Win32_Process
CreationDate               : 20160616153524.495967-360
CSCreationClassName        : Win32_ComputerSystem
CSName                     : RPTGEN01
Description                : java.exe
ExecutablePath             : C:\Pro

How do i get this to return just the count? or something that i can use for monitoring?

[Beandip408]

Okay i got it figured out. here is the script:

Code: Select all

$pCount = (Get-WmiObject Win32_Process -Filter "CommandLine like '%com.domain.reportgenerator.ReportGenerator%'").count

if ($pCount -eq 3) {
   write-host "OK - Found $pCount processes"
   exit 0
} if ($pCount -eq 2) {
   write-host "WARN - $pCount processes found"
   exit 1
} elseif ($pCount -lt 2) {
   write-host "CRITICAL - Less than 2 processes found"
   exit 2
}

and this is in the nagios config:

Code: Select all

define service {
        use                             critical-service
        hostgroup_name                  prod_rptgen
        service_description             Process - ReportGen
        check_command                   check_nrpe!check_rgstate!ShowAll!MaxWarn=2!MaxCrit=1
        notifications_enabled   1
}

Thanks for all of the help!

[ssax]

Awesome, thanks for posting that, I'm sure future visitors will find it very useful!

Are we okay to lock this and mark it as resolved?

[Beandip408]

Yep, this is resolved. Thanks.

[mcapra]

Closing and marking as resolved