Page 7 of 8
Re: Informix Monitoring
Posted: Tue Nov 20, 2018 5:45 pm
by tgriep
In the plugin, it is running the onstat command and then checking if it returns the string
On-Line
But the onstat command is not returning On-Line so the plugin returning an unknown state.
Can you login to the duey server and run the following commands and post the full output?
Code: Select all
export INFORMIXDIR=/usr/informix
export INFORMIXSERVER=dueynet
export ONCONFIG=onconfig
/usr/informix/bin/onstat -
echo $?
Is the old script still on the server and if so, can you use that script again?
This is the old script, if it exists, can you upload it?
Code: Select all
/usr/local/nagios/libexec/check_ifx.sh
Re: Informix Monitoring
Posted: Tue Nov 20, 2018 5:49 pm
by donnyforbes
Here is the output
Code: Select all
[root@duey ~]# export INFORMIXDIR=/usr/informix
[root@duey ~]# export INFORMIXSERVER=dueynet
[root@duey ~]# export ONCONFIG=onconfig
[root@duey ~]# /usr/informix/bin/onstat -
IBM Informix Dynamic Server Version 12.10.FC8W2WE -- On-Line (Prim) -- Up 24 days 16:47:15 -- 1269880 Kbytes
[root@duey ~]# echo $?
sorry I don't have the other script check_ifx.sh anymore the servers were wiped and reinstalled. It maybe in the previous case you and I worked on last time.... Again the scripts seem to be working from all 3 informix servers and also run from the NAGIOS server...
Re: Informix Monitoring
Posted: Wed Nov 21, 2018 9:15 am
by tgriep
Replace your current script with the contents below.
The check should return a OK state when the Informix server is On-Line and a Critical when it is not On-Line.
Code: Select all
#!/bin/bash
#
# check_ifx - plugin to validate that Informix is online # # parameters # 1) Informix directory # 2) Informix server
print_use(){
echo -e "USE: check_ifx informixdir informixserver\\n"
echo -e "\t informixdir = Informix directory"
echo -e "\t informixserver = server name"
}
STATE_OK=0
STATE_WARNING=1
STATE_CRITICAL=2
STATE_UNKNOWN=3
if [ $# -lt 2 ]
then
print_use
exit $STATE_UNKNOWN
fi
if [ $# -gt 2 ]
then
print_use
exit $STATE_UNKNOWN
fi
export INFORMIXDIR=$1
export INFORMIXSERVER=$2
export ONCONFIG=onconfig
result=`$INFORMIXDIR/bin/onstat -`
for cmd in $result
do
if [ $cmd = On-Line ]
then
echo $result
exit 0
fi
done
echo $result
exit 2
Re: Informix Monitoring
Posted: Wed Nov 21, 2018 9:28 am
by donnyforbes
Thanks for the reply. I did what you have suggested. Now I am seeing that it says critical even though informix is online see below.
I would have thought it would say "OK" since the 'onstat - ' command shows informix online.
Code: Select all
[root@huey libexec]# onstat -
IBM Informix Dynamic Server Version 12.10.FC8W2WE -- Updatable (Sec) -- Up 22 days 11:41:39 -- 1138808 Kbytes
[root@huey libexec]#
Re: Informix Monitoring
Posted: Wed Nov 21, 2018 9:32 am
by donnyforbes
Then what I notice is after a few minutes it goes back to "UNKNOWN" state see below. They way it should be working is if onstat - reports that it is up and running it should be green and say "OK", however if onstat - shows that it is down it should be RED and critical and then send alerts etc..
Re: Informix Monitoring
Posted: Wed Nov 21, 2018 10:22 am
by tgriep
From what I said earlier, the plugin is checking for the word On-Line in the data.
Your latest examples do not have the word On-Line it and that is why they are showing an non-OK state.
Take a look at line 36 of the script
This is the test the plugin is using to determine the state.
If you want it to check some other string, you would have to notify it.
The other check called "check for connection managers" Is that using the same plugin?
Are you sending different options to it? If so, what are they?
The question is, what does the ontstat command return in all of the various errors and which ones do you want to test against?
Re: Informix Monitoring
Posted: Wed Nov 21, 2018 10:47 am
by donnyforbes
again this is not complicated. when informix is up and running and you do an onstat - it will tell you in previous post.
if it is not running you will see this: This is from a test box.
when it is up and working it looks like this.
Code: Select all
[root@huey libexec]# onstat -
IBM Informix Dynamic Server Version 12.10.FC8W2WE -- Updatable (Sec) -- Up 22 days 13:10:17 -- 1138808 Kbytes
[root@huey libexec]#
[
Code: Select all
root@moe ~]# onstat -
shared memory not initialized for INFORMIXSERVER 'moenet'
[root@moe ~]#
You and I got this working a while ago so I am not sure what you are asking me.... I wish I good be some more help. Yes the connection managers is using the same plug in I just changed the command in it to be the onstat -g cmsm
This is what output - when it is up...
Code: Select all
[root@nag libexec]# /usr/local/nagios/libexec/check_nrpe -H 192.168.1.42 -c check_informix_cmsm -a '/usr/informix hueynet'
IBM Informix Dynamic Server Version 12.10.FC8W2WE -- Updatable (Sec) -- Up 21 days 11:26:03 -- 1138808 Kbytes Unified Connection Manager: rmca2_cm_hueynet Hostname: huey.sg1.allamericanasphalt.com CLUSTER rmca2_cluster_hq_hueynet LOCAL Informix Servers: rmcagrp SLA Connections Service/Protocol Rule rmca2_cm_hq_hueynet 13 rmcacmhqsoc/onsoctcp DBSERVERS=primary,HDR Failover Arbitrator: Active Arbitrator, Primary is up ORDER=SDS,HDR PRIORITY=1 TIMEOUT=10 SERVERSET rmca2_set_yard_hueynet Informix Servers: rmcagrp SLA Connections Service/Protocol Rule rmca2_cm_yard_hueynet 0 rmcacmyardsoc/onsoctcp DBSERVERS=(louienet),rmca2_cm_hq_hueynet SERVERSET rmca2_set_aix_hueynet Informix Servers: rmcagrp SLA Connections Service/Protocol Rule rmca2_cm_aix_hueynet 2379 rmcacmaixsoc/onsoctcp DBSERVERS=hueyaix,dueyaix Unified Connection Manager: rmca2_cm_dueynet Hostname: duey CLUSTER rmca2_cluster_hq_dueynet LOCAL Informix Servers: rmcagrp SLA Connections Service/Protocol Rule rmca2_cm_hq_dueynet 157 rmcacmhqsoc/onsoctcp DBSE
I wish I can be more help. I just remember we got this working together before.
Re: Informix Monitoring
Posted: Wed Nov 21, 2018 11:11 am
by tgriep
In your example string
IBM Informix Dynamic Server Version 12.10.FC8W2WE -- Updatable (Sec) -- Up 22 days 13:10:17 -- 1138808 Kbytes
The word On-Line is not in it but in it's place is "Updatable".
Is that a good status or a warning status or a critical status?
The other command, you added the -g CMSM to the command and it returns different data,
What in that data is considered good, bad, etc.....
Re: Informix Monitoring
Posted: Wed Nov 21, 2018 11:48 am
by donnyforbes
ok so change the script to say Updateable instead of On-line? Updateable means GOOD status means Informix is up and running.
If the command ran and return this for the output that means critical informix is DOWN.
This is a test box I used ....
Code: Select all
[root@moe ~]# onstat -
shared memory not initialized for INFORMIXSERVER 'moenet'
[root@moe ~]#
if the onstat - does not say UPDATABLE and says anything else that would mean critical...
The other onstat -g cmsm (test the connection mangers to see if they are up or down)
The output I sent you say there are up, when they are down they look like this ...
detected lost connection to Informix server (servername)
Does that make since?
Re: Informix Monitoring
Posted: Wed Nov 21, 2018 11:59 am
by donnyforbes
I changed the script from On-Line to Updatable and it seems to work. What I need to make sure is if it is not updateable and says anything else then it is critical and we get notified.