Script returns different result when ran locally versus nrpe

[cmerchant]

You could check the nagios log - /usr/local/nagios/var/nagios.log to see what Nagios is receiving before it gets parsed into the GUI. Is there a newline missing somewhere?

[rickwilson7425]

The nagios.log file doesn't show anything.

We are running Thruk and the thruk.log file just show the check being run, nothing about the return info:

[root@dennagp1 log]# cat thruk.log
[2014/11/03 08:54:59][dennagp1.jeppesen.com][INFO][Thruk.Controller.remote] started (1216)
[2014/11/03 08:55:02][dennagp1.jeppesen.com][INFO][Thruk.Controller.remote] 337 templates precompiled in 2.25s
[2014/11/03 08:55:10][dennagp1.jeppesen.com][INFO][Thruk.Controller.cmd] [rick.wilson][stage] cmd: COMMAND [1415030110] SCHEDULE_FORCED_SVC_CHECK;dencita1;Check Redis;1415030110
[2014/11/03 08:56:11][dennagp1.jeppesen.com][INFO][Thruk.Controller.cmd] [rick.wilson][stage] cmd: COMMAND [1415030171] SCHEDULE_FORCED_SVC_CHECK;dencita1;Check Redis;1415030171
[2014/11/03 09:25:42][dennagp1.jeppesen.com][INFO][Thruk.Controller.cmd] [rick.wilson][stage] cmd: COMMAND [1415031942] SCHEDULE_FORCED_SVC_CHECK;dencita1;Check Redis;1415031942
[2014/11/03 10:20:27][dennagp1.jeppesen.com][INFO][Thruk.Controller.cmd] [rick.wilson][stage] cmd: COMMAND [1415035227] SCHEDULE_FORCED_SVC_CHECK;dencita1;Check Redis;1415035227
[2014/11/03 10:22:26][dennagp1.jeppesen.com][INFO][Thruk.Controller.cmd] [rick.wilson][stage] cmd: COMMAND [1415035346] SCHEDULE_FORCED_SVC_CHECK;dencita1;Check Redis;1415035346

[abrist]

Your previous image did not attach correctly. What portion of the script is still missing the newline?

[rickwilson7425]

When we only get a 2 line return it looks fine.
When we get a 3rd line in the response it puts the first word of the 3rd return at the end of the 2nd line and the rest on the 3rd line. Like this:

REDIS Sentinel port 16379 on dencita1 is OK
REDIS Mobile port 6380 on dencita1 is OK REDIS
Nuxeo port 6379 on dencita1 is OK

The "REDIS" highlighted in red above should be the beginning of the 3rd line.

I attached a Word doc with a screen snippet so you can see better what I am talking about.

Redis Output.docx

(20.99 KiB) Downloaded 194 times

[abrist]

Well, the script sure as heck looks right. What happens if you add an extra newline to the end of the second (Mobile Port) echo?

[rickwilson7425]

Well, we got the multiple messages to appear in the GUI but are still having a problem with making it display each message on a separate line. We have tried adding padding characters to the messages but that only works if the browser is set to the right text size.

Here is what we have at this point:

Code: Select all

#!/bin/bash

################# Check SENTINEL Port #################

# Clint adds to make a single return option
sen_return=''
mobil_return=''
nuxeo_return=''
nuxeo_not_defined_error_return=''
mobile_not_defined_error_return=''
final_response_string=''
exit_value=0

HOSTNAME_REDIS=`hostname -s`

# Is the /etc/redis/rsentinel.conf file readable?
if [[ ! -r /etc/redis/rsentinel.conf ]];then
    # Failure because there is no rsentinel.conf file
    echo "Sentinel File requires; /etc/redis/rsentinel.conf doesn't exist on $HOSTNAME_REDIS: stopped check"
    exit 1
fi    
# Does the /etc/redis/rsentinel.conf contain a valid port line
if [[ ! `grep -P '^port ' /etc/redis/rsentinel.conf` ]];then
    # Failure because there is no port in rsentinel.conf file
    echo "NO PORT IDENTIFIED in /etc/redis/rsentinel.conf on $HOSTNAME_REDIS: stopped check"
    exit 1
fi  

# File and port exists; Gather ports (default sentinel port is 26379)
SEN_PORT=`grep -P '^port ' /etc/redis/rsentinel.conf | cut -f2 -d' '`
if [[ `/usr/local/bin/redis-cli -p $SEN_PORT PING | grep 'PONG'` ]];then
  # Success
  sen_return="Sentinel port $SEN_PORT on $HOSTNAME_REDIS is OK"
else
  # Failure
  sen_return="Sentinel port $SEN_PORT on $HOSTNAME_REDIS FAILED!!"
  exit_value=1
fi
  
################# Check MOBILE Port #################
# 
if [[ `find /apps/opt -maxdepth 1 -type d | grep /apps/opt/Tomcat_PWM` \
   || `find /apps/opt -maxdepth 1 -type d | grep /apps/opt/Tomcat_FDS` \
   || `find /apps/opt -maxdepth 1 -type d | grep /apps/opt/Tomcat_SYNC` \
   || `find /apps/opt -maxdepth 1 -type d | grep /apps/opt/Tomcat_REGISTRAR` ]];then

  # Does the /etc/redis/rsentinel.conf contain sentinel monitor MobileServices_Redis_Master line
  if [[ ! `grep -P '^sentinel monitor MobileServices_Redis_Master ' /etc/redis/rsentinel.conf ` ]];then
      # Failure because there is no MobileServices_Redis_Master in rsentinel.conf file
      mobile_not_defined_error_return="NO MobileServices_Redis_Master PORT IDENTIFIED in /etc/redis/rsentinel.conf on $HOSTNAME_REDIS: stopped check"
      exit_value=1
  else 
    MOBIL_PORT=`grep -P '^sentinel monitor MobileServices_Redis_Master ' /etc/redis/rsentinel.conf | cut -f5 -d' '`
    if [ -z $MOBIL_PORT ];then
      mobile_not_defined_error_return="WARNING $HOSTNAME_REDIS MOBILE REDIS NOT DEFINED IN SENTINEL."
      exit_value=1
    fi
    if [[ `/usr/local/bin/redis-cli -p $MOBIL_PORT PING | grep 'PONG'` ]];then
      # Success
       mobil_return="Mobile port $MOBIL_PORT on $HOSTNAME_REDIS is OK"
    else
      # Failure
       mobil_return="Mobile port $MOBIL_PORT on $HOSTNAME_REDIS FAILED!!"
      exit_value=1
    fi
  fi
fi
################### Check NUXEO Port #################

if [[ `find /apps/opt -maxdepth 1 -type d | grep /apps/opt/nuxeo-jdm-server` ]];then
  # Does the /etc/redis/rsentinel.conf contain sentinel monitor MobileServices_Redis_Master line
  if [[ ! `grep -w Nuxeo_Redis_Master /etc/redis/rsentinel.conf ` ]];then
      # Failure because there is no Nuxeo_Redis_Master in rsentinel.conf file
      nuxeo_not_defined_error_return="NO Nuxeo_Redis_Master PORT IDENTIFIED in /etc/redis/rsentinel.conf on $HOSTNAME_REDIS: stopped check"
      exit_value=1
  else 
    NUXEO_PORT=`grep -w Nuxeo_Redis_Master /etc/redis/rsentinel.conf | cut -f5 -d' '`
    if [ -z $NUXEO_PORT ];then
      nuxeo_not_defined_error_return="WARNING $HOSTNAME_REDIS NUXEO REDIS NOT DEFINED IN SENTINEL."
      exit_value=1
    fi
    if [[ `/usr/local/bin/redis-cli -p $NUXEO_PORT PING | grep 'PONG'` ]];then
      # Success
       nuxeo_return="Nuxeo port $NUXEO_PORT on $HOSTNAME_REDIS is OK"
    else
      # Failure
      nuxeo_return="Nuxeo port $NUXEO_PORT on $HOSTNAME_REDIS FAILED!!"
      exit_value=1
    fi
  fi  
fi
  
# Evaluate and return pretty printed output

final_response_string='';
for i in "$sen_return" "$mobil_return" "$nuxeo_return" "$nuxeo_not_defined_error_return" "$mobile_not_defined_error_return";do 
  if [[ -z "$i" ]];then continue;fi
  export final_response_string="$final_response_string"`echo "$i" | perl -nle'$x=( length($_) > 45 ) ? ( 43 - (length($_) % 45)) : ( 43 - length($_) );print "$_" . ". "x${x};'`
done

echo "$final_response_string"

exit $exit_value

Where are we going wrong with getting the \n recognized by Nagios?

[sreinhardt]

I'm sorry I have to retract my previous statement on \n being needed for active checks, I walked through this with one of our core developers yesterday to confirm. This is ONLY needed for things submitted via passive checks or the command pipe, as newlines are used for separating characters. This is not the case with active checks, and actual newlines should be used as you originally thought. So, the outlines you need to follow currently are(this is how you would actually send it back to nagios):

Just single line text output:
[standard status output]

Single line w/ perfdata:
[standard status output] | [performance data]

Multi-line w/0 perfdata:
[standard status output]
[long service output]

Multi-line w/ perfdata:
[standard status output]
[long service output] | [performance data]
OR
[standard status output]
[long service output]
| [performance data]

It's stated in the return information guidelines that something like:
[standard status output] | [performance data]
[long service output] | [performance data]

Should result in both sets of performance data being reaped correctly. This is presently not the case, and all output needs to be implemented with perfdata as the final output, whether on the same line or a newline from other output. I see that your output is presently only fitting certain size screens. Seems to me this may be due to it all being on one actual line(not sure of how that perl works), when as it fits a screen properly, it wraps as it should, and both core and thruk likely artificially limit the are that text can fit depending on your window size.

The one thing I need to point out, is since you are looking to do this all in the normal status output and not have to use long service output, which using newlines forces you into. How about using the html character for newline(<br />)? Provided thruk displays this correctly, it should both keep you within standard service output, and allow a multi-line display for you.

[rickwilson7425]

Well, it seems I am just going in circles on this. I have attached a Word doc with screen shots of what I am getting.

Here is the script I am experimenting with:

Code: Select all

#!/bin/bash

################# Check SENTINEL Port #################

# Clint adds to make a single return option
sen_return=''
mobil_return=''
nuxeo_return=''
nuxeo_not_defined_error_return=''
mobile_not_defined_error_return=''
final_response_string=''
exit_value=0

HOSTNAME_REDIS=`hostname -s`

# Gather ports (default sentinel port is 26379)
SEN_PORT=`grep -P '^port ' /etc/redis/rsentinel.conf | cut -f2 -d' ' | sort -u | tail -1`
  if [[ `/usr/local/bin/redis-cli -p $SEN_PORT PING | grep 'PONG'` ]];then
    # Success
    sen_return="Sentinel port $SEN_PORT on $HOSTNAME_REDIS is OK. "
  else
    # Failure
    sen_return="Sentinel port $SEN_PORT on $HOSTNAME_REDIS FAILED!!"
    exit_value=1
  fi
  
################# Check MOBILE Port #################

if [[ `find /apps/opt -maxdepth 1 -type d | grep /apps/opt/Tomcat_JDM` \
   || `find /apps/opt -maxdepth 1 -type d | grep /apps/opt/Tomcat_PWM` \
   || `find /apps/opt -maxdepth 1 -type d | grep /apps/opt/Tomcat_FDS` \
   || `find /apps/opt -maxdepth 1 -type d | grep /apps/opt/Tomcat_SYNC` \
   || `find /apps/opt -maxdepth 1 -type d | grep /apps/opt/Tomcat_REGISTRAR` ]];then

  MOBIL_PORT=`grep 'MobileServices_Redis_Master ' /etc/redis/rsentinel.conf  | grep 'monitor' | cut -f5 -d' ' | sort -u | tail -1`
  if [ -z $MOBIL_PORT ];then
    mobile_not_defined_error_return="WARNING $HOSTNAME_REDIS MOBILE REDIS NOT DEFINED IN SENTINEL."
    exit_value=1
  fi
  if [[ `/usr/local/bin/redis-cli -p $MOBIL_PORT PING | grep 'PONG'` ]];then
    # Success
     mobil_return="Mobile port $MOBIL_PORT on $HOSTNAME_REDIS is OK. "
  else
    # Failure
     mobil_return="Mobile port $MOBIL_PORT on $HOSTNAME_REDIS FAILED!!"
    exit_value=1
  fi
fi

################### Check NUXEO Port #################

if [[ `find /apps/opt -maxdepth 1 -type d | grep /apps/opt/nuxeo-jdm-server` ]];then
  NUXEO_PORT=`grep -w Nuxeo_Redis_Master /etc/redis/rsentinel.conf | grep 'monitor' | cut -f5 -d' ' | sort -u | tail -1`
  if [ -z $NUXEO_PORT ];then
    nuxeo_not_defined_error_return="WARNING $HOSTNAME_REDIS NUXEO REDIS NOT DEFINED IN SENTINEL."
    exit_value=1
  fi
  if [[ `/usr/local/bin/redis-cli -p $NUXEO_PORT PING | grep 'PONG'` ]];then
    # Success
     nuxeo_return="Nuxeo port $NUXEO_PORT on $HOSTNAME_REDIS is OK. "
  else
    # Failure
    nuxeo_return="Nuxeo port $NUXEO_PORT on $HOSTNAME_REDIS FAILED!!"
    exit_value=1
  fi
fi  
  
# Evaluate and return pretty printed output

final_response_string='';
# export final_response_string="${sen_return} ${mobil_return} ${nuxeo_return} ${nuxeo_not_defined_error_return} ${mobile_not_defined_error_return}"
for i in "$sen_return" "$mobil_return" "$nuxeo_return" "$nuxeo_not_defined_error_return" "$mobile_not_defined_error_return";do
if [[ -z "$i" ]];then continue;fi
# export final_response_string="$final_response_string"`echo "$i" | perl -nle'$x=( length($_) > 45 ) ? ( 44 - (length($_) % 45)) : ( 44 - length($_) );print "$_" . ". "x${x};'`
# export final_response_string="$sen_return $nuxeo_return"
done
printf '%b\n%s' "$sen_return" "$nuxeo_return"
# done

# echo "$final_response_string"

exit $exit_value

Please ignore all the commented out sections, they are just different experiments that didn't work either.

What would be the advantages/disadvantages to using the long service output?

BTW - the <br \> didn't work at all.

[rickwilson7425]

Redis Display Problem.docx

(44.75 KiB) Downloaded 212 times

Sorry, I forgot to attach the Word doc.

[rickwilson7425]

Sorry again, I sent the Word doc because the result display in the GUI is blue, instead of the normal black, and I was wondering if that is a clue as to what is going wrong.