Need to Monitor Informix

[tgriep]

Login to the remote server, run the following and post the output.

Code: Select all

su - nagios
bash -x /usr/local/nagios/libexex/check_ifx.sh
/usr/informix/bin/onstat -

[ssax]

Please send the entire sanitize output of these commands:

Try it as the nagios user (run these exact commands, including the minus in su - nagios), if this doesn't work, it won't work through NRPE:

Code: Select all

su - nagios
/usr/informix/bin/onstat -

My question is do you need to su to your informix user in order to run the onstat - command? That is what we needed to do because all of the permissions/envars/etc were based around that when I was an informix server admin (this was 5 or 6 years ago so I am by no means an Informix expert, we also ran it on AIX so it's a bit different).

I am starting from scratch on the debugging so that we can get this resolved for you, I will test in my environment to make sure it works and then we can focus on yours (to debug the issue), first we need to validate that it actually works the way we are trying but because we do not have any informix systems here we need the output to test.

Thank you

[ssax]

Please run tgriep's commands as well.

Thank you

[donnyforbes]

Login to the remote server, run the following and post the output.

Code: Select all

su - nagios
bash -x /usr/local/nagios/libexex/check_ifx.sh
/usr/informix/bin/onstat -

here is what I get
[nagios@huey ~]$ bash -x /usr/local/nagios/libexex/check_ifx.sh
bash: /usr/local/nagios/libexex/check_ifx.sh: No such file or directory
[nagios@huey ~]$

[donnyforbes]

Login to the remote server, run the following and post the output.

Code: Select all

su - nagios
bash -x /usr/local/nagios/libexex/check_ifx.sh
/usr/informix/bin/onstat -

here is the output when I ran it with the ./check_ifx.sh (this is the only way it runs)

Code: Select all

[nagios@huey libexec]$ ./check_ifx.sh
CRITICAL
[nagios@huey libexec]$

Here is the output of the other command

Code: Select all

[nagios@huey libexec]$ /usr/informix/bin/onstat -
shared memory not initialized for INFORMIXSERVER '<NULL>'
[nagios@huey libexec]$

As root here is the output...I usually run this command as root

[

Code: Select all

root@huey libexec]# /usr/informix/bin/onstat -
IBM Informix Dynamic Server Version 12.10.FC8W2WE -- On-Line (Prim) -- Up 3 days 02:40:56 -- 1034088 Kbytes

[donnyforbes]

you quesiton?

My question is do you need to su to your informix user in order to run the onstat - command?

It can run as informix or root.

Code: Select all

[root@huey libexec]# su informix
[informix@huey libexec]$ onstat -

IBM Informix Dynamic Server Version 12.10.FC8W2WE -- On-Line (Prim) -- Up 3 days 02:43:41 -- 1034088 Kbytes

[informix@huey libexec]$

Code: Select all

[root@huey libexec]# onstat -

IBM Informix Dynamic Server Version 12.10.FC8W2WE -- On-Line (Prim) -- Up 3 days 02:44:01 -- 1034088 Kbytes

[root@huey libexec]#

I usually run the onstat - command via root but both will work. Hope this helps.

[ssax]

As root, what is the output of this command:

Code: Select all

env

Then send the output of these commands and see if it works:

Code: Select all

su - nagios
sudo /usr/informix/bin/onstat -
bash -x ./check_ifx.sh

Thank you

[donnyforbes]

As root, what is the output of this command:

Code: Select all

env

Then send the output of these commands and see if it works:

Code: Select all

su - nagios
sudo /usr/informix/bin/onstat -
bash -x ./check_ifx.sh

Thank you

Here's the out put

Code: Select all

[root@nag sbin]# env
XDG_SESSION_ID=165710
HOSTNAME=nag
TERM=xterm
SHELL=/bin/bash
HISTSIZE=1000
SSH_CLIENT=192.168.1.161 56512 22
OLDPWD=/usr/local/nagiosxi
SSH_TTY=/dev/pts/0
USER=root
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:
MAIL=/var/spool/mail/root
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
PWD=/usr/local/nagios/sbin
LANG=en_US.UTF-8
HISTCONTROL=ignoredups
SHLVL=1
HOME=/root
LOGNAME=root
PD_SERVICE_KEY=2c83d6596db141879a2c2a24a46f0de3
SSH_CONNECTION=192.168.1.161 56512 192.168.1.186 22
LESSOPEN=||/usr/bin/lesspipe.sh %s
XDG_RUNTIME_DIR=/run/user/0
_=/usr/bin/env
[root@nag sbin]#

see below

Code: Select all

[nagios@nag ~]$ whoami
nagios
[nagios@nag ~]$

Code: Select all

[nagios@nag ~]$ su /usr/informix/bin/onstat -
su: user /usr/informix/bin/onstat does not exist

Code: Select all

[nagios@nag ~]$ bash -x ./check_ifx.sh
bash: ./check_ifx.sh: No such file or directory
[nagios@nag ~]$

Sorry wrong server .. Let me try again....

[donnyforbes]

Env

Code: Select all

[root@huey libexec]# env
XDG_SESSION_ID=2531
HOSTNAME=huey.sg1.allamericanasphalt.com
SELINUX_ROLE_REQUESTED=
TERM=xterm
SHELL=/bin/bash
HISTSIZE=1000
SSH_CLIENT=192.168.1.161 53637 22
INFORMIXDIR=/usr/informix
SELINUX_USE_CURRENT_RANGE=
SSH_TTY=/dev/pts/0
USER=root
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:
ONCONFIG=onconfig
MAIL=/var/spool/mail/root
PATH=/usr/informix/bin:/usr/informix/extend/krakatoa/jre/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
PWD=/usr/local/nagios/libexec
LANG=en_US.UTF-8
SELINUX_LEVEL_REQUESTED=
SSH_ASKPASS=/usr/libexec/openssh/lxqt-openssh-askpass
HISTCONTROL=ignoredups
INFORMIXSERVER=hueynet
SHLVL=1
HOME=/root
INFORMIXSQLHOSTS=/usr/informix/etc/sqlhosts
LOGNAME=root
SSH_CONNECTION=192.168.1.161 53637 192.168.1.42 22
LESSOPEN=||/usr/bin/lesspipe.sh %s
XDG_RUNTIME_DIR=/run/user/0
_=/usr/bin/env
OLDPWD=/usr/local/nagios
[root@huey libexec]#

Code: Select all

[root@huey libexec]# su - nagios
Last login: Thu Jul 13 12:42:17 PDT 2017 on pts/0
[nagios@huey ~]$

Code: Select all

[nagios@huey ~]$ su /usr/informix/bin/onstat -
su: user /usr/informix/bin/onstat does not exist
[nagios@huey ~]$

Code: Select all

[nagios@huey ~]$ bash -x ./check_ifx.sh
bash: ./check_ifx.sh: No such file or directory
[nagios@huey ~]$

[tgriep]

We found that the Informix onstat command needs to have some environment set as the command uses them when it is executed.
The reason the root and the informix user works, the variables are set but the nagios user account does not.

So edit the check_ifx.sh script and change it to the following

Code: Select all

#!/bin/bash
#
# check_ifx - plugin to validate that informix is onlinea # # # parameters # 1) Informix directory # 2) Informix server
export INFORMIXDIR=/usr/informix
export INFORMIXSERVER=hueynet
export ONCONFIG=onconfig

result=`/usr/informix/bin/onstat -`

for cmd in $result
do
  if [ $cmd = On-Line ]
  then
	echo $result
	exit 0
  fi
done
echo "CRITICAL"
exit 2

Save it out and see if it works by running

Code: Select all

su - nagios
/usr/local/nagios/libexec/check_ifx.sh

If you use this script on another server, you would have to edit the following lines if needed.
The INFORMIXSERVER line for sure will have to be edited.

Code: Select all

export INFORMIXDIR=/usr/informix
export INFORMIXSERVER=hueynet
export ONCONFIG=onconfig