Switch port status

[devildog31415]

Nagiosxi 2014r1.1 (virtual, downloaded from nagios)

Our core Cisco switch has several ports that are either always up, or always down.

Up ports:
I use the "check_xi_service_ifoperstatus" command and get expected results

Down ports:
I use the "negate" plugin as outlined here: http://assets.nagios.com/downloads/nagi ... Plugin.pdf

I just ran into an issue where a port is now unused and instead of just leaving the port available but unplugged, the network guys put the port in "administrative down" and now regardless of whether I check it with "check_xi_service_ifoperstatus" or with "negate_port_status" the port is sitting with state:
WARNING: Interface GigabitEthernet9/26 (index 190) is administratively down.

How can I address this since it is absolutely appropriate for an unused port to be in "administrative down" state?

[tmcdonald]

Wrapper scripts to the rescue!

I would write a simple wrapper that calls the appropriate plugin with all the arguments, and then if the exit code is a WARNING and the status message contains "is administratively down" it would instead show as OK. I'd be happy to help with this if you need it.

[devildog31415]

That would be HUGELY appreciated Trevor!
You were invaluable the other day on the check_nt_wrapper.sh script.

[tmcdonald]

You're quite welcome for that.

So for starters, let's define the problem. You want the following states in Nagios for the associated events:

Code: Select all

Port is UP            = OK
Port is DOWN          = CRITICAL
Port is always DOWN   = OK
Port is in Admin Down = OK

I assume that the Always Down ports should be OK based on your usage of negate, and instead of a WARNING you want OK for the Admin Down ports. And although it wasn't stated specifically, I also assume you want a CRITICAL when a normally-UP port goes DOWN.

Is this accurate? It's important to cover all the bases.

[devildog31415]

Actually, based on the ability to use "admin down" we could stop using the "negate" check because I can have the network team put any port that should be down in "admin down" and then use the following logic?

Then the only check needed would be: check_xi_service_ifoperstatus
Port is UP = OK
Port is DOWN = CRITICAL
Port is ADMIN DOWN = OK


That way any port that SHOULD be down would be changed by the network team and wouldn't require any config changes on NAGIOS and I wouldn't need the negate command at all.

Is that possible?

[tmcdonald]

Take a look at how bash scripts handle searching for strings:

Code: Select all

#!/bin/sh

result=`$1`
exit_code=$?

if [[ "$result" =~ "administratively down" ]]
  then
    echo "Found!"
  else
    echo "Not found..."
fi

echo $exit_code

If you save this as check_ifoperstatus_wrapper and execute it (you may have to chmod +x it first) with the check_ifoperstatus command as the only argument, it will search the output to see if "administratively down" exists within it. So something like this (run from the /usr/local/nagios/libexec/ directory):

Code: Select all

./check_ifoperstatus_wrapper "<check_ifoperstatus command>"

Depending on the command you run and the interface status, it will either alert that it has or has not found the string. You can test this with the following:

Code: Select all

./check_ifoperstatus_wrapper "echo 'administratively down'"
./check_ifoperstatus_wrapper "echo 'administratively awesome'"

Essentially the script runs as a command whatever you type after it. In this example it runs a simple echo command, with the first string being a match and the second not.

Going off of this it should be pretty easy to mess around with bash and write wrappers for just about anything. You can add else clauses to the check for strings, and the exit code of your wrapped script can be determined with the $? variable used above. So if the exit code is 0, it was an OK result. If the code is 1 it is a WARNING and 2 is a CRITICAL.

Let me know if anything needs clarification.

[devildog31415]

struggling... I marked up my bash heavily (more than necessary) just as a sanity check and to make sure you can see what I am thinking and correct accordingly. I think that the response from the core script has special characters and is breaking the process???

if I do:
./check_ifoperstatus -H 192.168.169.1 -C password-k 190 -v 2

the response is:
WARNING: Interface GigabitEthernet9/26 (index 190) is administratively down.

If I try to run my wrapper script by doing:
./check_ifoperstatus_wrapper_admin-down.sh 192.168.169.1 password 190 2

the response is:
./check_ifoperstatus_wrapper_admin-down.sh: line 10: if[[ WARNING:: command not found
./check_ifoperstatus_wrapper_admin-down.sh: line 11: syntax error near unexpected token `then'
./check_ifoperstatus_wrapper_admin-down.sh: line 11: ` then'


HERE is the wrapper script "check_ifoperstatus_wrapper_admin-down.sh"

Code: Select all

#!/bin/bash

HOSTADDRESS=$1  #Host address
SNMPSTRING=$2   #snmp ro string
SNMPPORTID=$3   #switch port to query
SNMPVERSION=$4  #snmp version

#run core check and accept it's response for parsing
CHECKIFOPERSTATUSRESPONSE=`/usr/local/nagios/libexec/check_ifoperstatus -H $HOSTADDRESS -C $SNMPSTRING -k $SNMPPORTID -v $SNMPVERSION`
if[[ $CHECKIFOPERSTATUSRESPONSE == *administratively* ]]        #check response for the word "administratively"
        then
                ACTUALSTATE=OK  #set variable to replace "WARNING" with "OK" in the string
                NEWECHO="${CHECKIFOPERSTATUSRESPONSE/WARNING/$ACTUALSTATE}""    #replace WARNING with OK
                echo $NEWECHO   #output modified response
                exit 0  #exit with "OK" status
        else
                if[[ "$CHECKIFOPERSTATUSRESPONSE" == *up* ]]    #check response for the word "up"
                        then
                                echo $CHECKIFOPERSTATUSRESPONSE
                                exit 0  #exit with "OK" status
                        else
                                if[[ "$CHECKIFOPERSTATUSRESPONSE" == *down* ]]  #check response for the word "down"
                                        then
                                                echo $CHECKIFOPERSTATUSRESPONSE
                                                exit 2  #exit with "CRITICAL" status
                                fi
                fi
fi
# if no criteria met, echo response from core script and set exit code to "WARNING"
echo $CHECKIFOPERSTATUSRESPONSE
exit 1

[tmcdonald]

A few things to clear up:

Bash is pretty picky about the spacing with certain commands, in particular the brackets for your if statements:

http://stackoverflow.com/questions/2237 ... ash-script

Stick a space after your "if" and before the bracket. In addition, you'll need a semi-colon at the end of the if line as demonstrated in the StackOverflow link I posted.

You can also clean up the if/else logic a bit with the "elif" (else+if) command: http://www.thegeekstuff.com/2010/06/bas ... -examples/

Usually that's optional but it can make things a bit easier to read.

[devildog31415]

I think I have cleaned it up following your guidance.. thank you.
Below is the code I have now but I still have some odd errors:

[root@log_book libexec]# ./check_ifoperstatus_wrapper_admin-down.sh 192.168.169.1 Password 190 2
./check_ifoperstatus_wrapper_admin-down.sh: line 19: unexpected EOF while looking for matching `"'
./check_ifoperstatus_wrapper_admin-down.sh: line 27: syntax error: unexpected end of file

Still guessing that it has to do with the contents of the string being passed back with special characters in it?
[root@log_book libexec]# ./check_ifoperstatus -H 192.168.169.1 -C Password-k 190 -v 2
WARNING: Interface GigabitEthernet9/26 (index 190) is administratively down.

Code: Select all

#!/bin/bash

HOSTADDRESS=$1  #Host address
SNMPSTRING=$2   #snmp ro string
SNMPPORTID=$3   #switch port to query
SNMPVERSION=$4  #snmp version

CHECKIFOPERSTATUSRESPONSE=`/usr/local/nagios/libexec/check_ifoperstatus -H $HOSTADDRESS -C $SNMPSTRING -k $SNMPPORTID -v $SNMPVERSION`
if [[ $CHECKIFOPERSTATUSRESPONSE == *administratively* ]];
then
        ACTUALSTATE=OK
        NEWECHO="${CHECKIFOPERSTATUSRESPONSE/WARNING/$ACTUALSTATE}""
        echo $NEWECHO
        exit 0
elif [[ "$CHECKIFOPERSTATUSRESPONSE" == *up* ]];
        then
                echo $CHECKIFOPERSTATUSRESPONSE
                exit 0
elif [[ "$CHECKIFOPERSTATUSRESPONSE" == *down* ]];
        then
                echo $CHECKIFOPERSTATUSRESPONSE
                exit 2
else
        echo $CHECKIFOPERSTATUSRESPONSE
        exit 1
fi

[tmcdonald]

You have an extra quote on line 12:

Code: Select all

        NEWECHO="${CHECKIFOPERSTATUSRESPONSE/WARNING/$ACTUALSTATE}""