SNMP not sending output correctly

[mxthunder]

I am running a custom bash script on nagios 5.4.9 that sends an SNMP trap to our NNMi server. The output is not coming across correctly on one server. I use another script that sends the output across just fine. What could the difference be?
Log file and partial script below.


Thanks
Jeff

=========
RUNNING COMMAND: /usr/bin/snmptrap -v 2c -c FDSCOMMON 11.48.30.40:162 '' NAGIOS-NOTIFY-MIB::nSvcEvent nSvcHostname s "lp000xsesn0017" nSvcDesc s "storeops_logparser2_IWM_SHIPPABLE_RTF_test_ignore" nSvcStateID i 1 nSvcOutput s "Results=1"
[service] => storeops_logparser2_IWM_SHIPPABLE_RTF_test_ignore
[service] => storeops_logparser2_IWM_SHIPPABLE_RTF_test_ignore
=========
if [ "$LogResult" -ge "1" ];then

RESULT=$STATE_WARNING
msg_text=`sshpass -p$PASS ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=310 $USER@$HOST "$COMMAND2"`
echo "Results=$RESULT"
echo "$msg_text"
exit $RESULT
fi
=========
2nd script
=========

if [ -z "$LogResult" ];then
echo "File is empty or unreadable."
RESULT=$STATE_UNKNOWN
echo "Results=$RESULT"
exit $RESULT
else



if [ "$LogResult" -eq "0" ];then

RESULT=$STATE_OK
msg_text="DNI Health report is good."
echo "Results=$RESULT"
echo "$msg_text"
exit $RESULT
fi

if [ "$LogResult" -ge "1" ];then
RESULT=$STATE_WARNING
msg_text="DNI Health report is showing BAD."
echo "Results=$RESULT"
echo "$msg_text"
exit $RESULT
fi


fi


======================================


RUNNING COMMAND: /usr/bin/snmptrap -v 2c -c FDSCOMMON 11.48.30.40 '' NAGIOS-NOTIFY-MIB::nSvcEvent nSvcHostname s "esu1l278" nSvcDesc s "MFT DNI Log Health_V4" nSvcStateID i 3 nSvcOutput s "File is empty or unreadable. Results=3"
RUNNING COMMAND: /usr/bin/snmptrap -v 2c -c FDSCOMMON 11.48.30.40 '' NAGIOS-NOTIFY-MIB::nSvcEvent nSvcHostname s "esu1l278" nSvcDesc s "MFT DNI Log Health_V4" nSvcStateID i 3 nSvcOutput s "File is empty or unreadable. Results=3"
[service] => MFT DNI Daemon Service
[service] => MFT DNI Daemon Service
[service] => MFT DNI Daemon Service
[service] => MFT DNI Daemon Service
[service] => MFT DNI Daemon Service
[serviceoutput] => s95d2z2 DNI Daemon Service is running
[longserviceoutput] => s95d2z2 DNI Daemon Service is running
[service] => MFT DNI Daemon Service

[scottwilkerson]

One of them is showing the host as 11.48.30.40 the other 11.48.30.40:162 (including port 162)

I'm not sure which is working for you but this is a difference I am seeing from what you posted

[tgriep]

Can you provide the full bash script so we can view all of it.
How are the commands created in the Nagios configuration files that are used to run the script? Post them as well.

What does the bad data look like and how should it look?
Provide an example.

[mxthunder]

Script that works:
============================================

Code: Select all

STATE_OK=0
STATE_WARNING=1
STATE_CRITICAL=2
STATE_UNKNOWN=3
HOST=$1
FILE=$2
KEYWORD=$3
USER=""
PASS=""
COMMAND="if [ -r "$FILE" ]; then cat $FILE | grep $KEYWORD | wc -l ; fi"
#COMMAND='tail -6 /tmp/logtext.txt | grep "ERROR" | wc -l'
#COMMAND='top -b -n1 | grep Cpu | awk NF=2 | cut -d : -f 2 | cut -d . -f 1'
#COMMAND2='top -b -n1 | grep Cpu | awk NF=2 | cut -d : -f 2 | cut -d . -f 1'

LogResult=`sshpass  -p$PASS ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=310 $USER@$HOST "$COMMAND"`

if [ -z "$LogResult" ];then
	echo "File is empty or unreadable."
	RESULT=$STATE_UNKNOWN
	echo "Results=$RESULT"
	exit $RESULT
else



	if [ "$LogResult" -eq "0" ];then
	
		RESULT=$STATE_OK
		msg_text="DNI Health report is good."
		echo "Results=$RESULT"
		echo "$msg_text"
		exit $RESULT
	fi

	if [ "$LogResult" -ge "1" ];then
		RESULT=$STATE_WARNING
		msg_text="DNI Health report is showing BAD."
		echo "Results=$RESULT"
		echo "$msg_text"
		exit $RESULT
	fi


fi

============================================
script that does not work:

Code: Select all

STATE_OK=0
STATE_WARNING=1
STATE_CRITICAL=2
STATE_UNKNOWN=3
HOST=$1
FILE=$2
KEYWORD=$3
KEYWORD2=$4
USER=""
PASS=""
COMMAND="if [ -r "$FILE" ]; then cat $FILE | grep $KEYWORD | grep $KEYWORD2 | wc -l ; fi"
COMMAND2="if [ -r "$FILE" ]; then cat $FILE | grep $KEYWORD | grep $KEYWORD2 ; fi"
LogResult=`sshpass  -p$PASS ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=310 $USER@$HOST "$COMMAND"`
#LogResult=`/usr/local/nagios/sshpass-1.05/sshpass  -p$PASS ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=310 $USER@$HOST "$COMMAND"`

if [ -z "$LogResult" ];then
	echo "File is empty or unreadable."
	RESULT=$STATE_UNKNOWN
	echo "Results=$RESULT"
	exit $RESULT

else

	if [ "$LogResult" -eq "0" ];then
	
		RESULT=$STATE_OK
		msg_text="No errors found."
		echo "Results=$RESULT"
		echo "$msg_text"
		exit $RESULT
	fi

	if [ "$LogResult" -ge "1" ];then
	
		RESULT=$STATE_WARNING
		msg_text=`sshpass  -p$PASS ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=310 $USER@$HOST "$COMMAND2"`
		echo "Results=$RESULT"
		echo "tesssssssssssssssssssssssssst"
		exit $RESULT
	fi

fi

===============================================
attached screenshots of good and bad output.

[mxthunder]

.

[tgriep]

I guess that you are trying to get the output of this command

Code: Select all

msg_text=`sshpass -p$PASS ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=310 $USER@$HOST "$COMMAND2"`

and have it shown in the output, correct?

If you run the script manually, what does it show on the output?

If the output is very large, it could be getting truncated.

Also, the data is getting printed on a separate line which maybe why it is lost.

Try editing the script and change this line from

Code: Select all

echo "Results=$RESULT"

to

Code: Select all

echo "Results=$RESULT $msg_text"

That way the data will all print on one line.

[mxthunder]

Thanks. The output appears to be working when run manually.
Is there a way I can prevent this from being truncated or is there a hard limit to the output length?

[tgriep]

When a plugin runs and returns the data on more than one line, Nagios considers the first line to be the service output and all of the other lines are considered as long service output.

$SERVICEOUTPUT$ The first line of text output from the last service check (i.e. "Ping OK").
$LONGSERVICEOUTPUT$ The full text output (aside from the first line) from the last service check.

In the snmptrap command, you are only sending the nSvcOutput data which is only the first line of the plugin output.

That is why I suggested adding the $msg_text variable to be put on the same line so the plugin outputs all of the data on one line.
That way it should be sent over in the snmptrap command.