PROCS WARNING: 151 processes

[ttung888]

***** Nagios XI Alert *****

Nagios has detected this service has stabilized. Notifications will now be sent out as needed on a normal basis.

Notification Type: FLAPPINGSTOP

Service: Total Processes
Host: ******
Address: **********
State: WARNING
Info:
PROCS WARNING: 151 processes
Date/Time: 2013-05-30 13:44:21


anyone knows what is that meant?
how can you see the detail of the 151 processes?
thanks

[abrist]

The detail of the processes are not supported by the check. You could make an event handler to run a script through nrpe (if it is a remote host) to dump a "ps" command to a file in /tmp.

[ttung888]

it's remote host, how can you run a script through nrpe?
thanks

[abrist]

This can get a bit complex, so bare with me.
This example uses nrpe on a linux system. You will need to have nrpe installed on the remote host to deploy this event handler.
You need to create a script that will output a process list to a file. For example:

--start remote host configuration--
Create a script file on the remote system:

Code: Select all

nano /usr/local/nagios/libexec/ps.sh

Copy the following code into the file:

Code: Select all

#!/bin/bash
DATE=$(date +"%y-%m-%d-%T")
ps -aef > /tmp/process_list_$DATE

Make it executable:

Code: Select all

chmod +x /usr/local/nagios/libexec/ps.sh

Now create a command definition in the remote host's nrpe.cfg file:

Code: Select all

nano /usr/local/nagios/etc/nrpe.cfg

Add the following line to the bottom of the file:

Code: Select all

command[list_procs_event]=/usr/local/nagios/libexec/ps.sh

Restart nrpe:

Code: Select all

service xinetd restart

--end remote host configuration--

--start XI configuration--
Now add a misc command to XI to be used as an event handler. Go to Configure --> Core Config Manager --> Commands --> Add new.
For Command name, enter:

Code: Select all

list_procs_event

For command line, enter:

Code: Select all

$USER1$/check_nrpe -H $HOSTADDRESS$ -t  30 -c list_procs_event

Change command type dropdown to:

Code: Select all

misc command

Save.
Now lets add the handler to the service: Go to Configure --> Core Config Manager --> Services. Search and edit the service in question.
Under the tab "Check settings", choose "list_procs_event" in the "Event Handler" dropdown. Bullet "On" for the "Event handler enabled" field.
Save and "apply configuration".
--end XI configuration--

Now whenever the service check changes state (like a warning or critical), it will fire off the event which is a command that runs an nrpe check to the remote host to run the nrpe command "list_procs_event" which in turn runs the "ps.sh" script on the remote host. This script will create a time stamped file in /tmp named "process_list_" with the date appended on the end. When you get a warning or critical, you can pull a terminal to the remote host and check the contents of the /tmp/process_list_[date] file.

On the remote host:

Code: Select all

cat /tmp/process_list_*

[ttung888]

how can we change the threshold from 150 to 200 for one monitored server?

[abrist]

how can we change the threshold from 150 to 200 for one monitored server?

You can do this by changing the command definition in the remote hosts nrpe.cfg.

[ttung888]

what is the path to nrpe.cfg? what need to be changed?

thanks

[abrist]

I presume that nrpe is installed on the remote host (if not, what agent are you using)? If you used our agent, the path to nrpe.cfg is:

Code: Select all

/usr/local/nagios/etc/nrpe.cfg

You may also have a common.cfg file that needs to be edited as well.

[ttung888]

I have installed the nagios agent on the Ubuntu 12.04 server, and everything seems fine except the CPU status shows unknown (iostat not found or is not executable by the nagious user).
anyone knows how to fix it?


thanks

[slansing]

You will need to install the "iostat" package via your package manager, for ubuntu you are most likely using "apt get"