Need to Monitor Informix

[donnyforbes]

Made this change and it seemed to work running the command.

Code: Select all

[root@huey libexec]# su - nagios
Last login: Thu Jul 13 15:29:40 PDT 2017 on pts/0
[nagios@huey ~]$ /usr/local/nagios/libexec/check_ifx.sh
IBM Informix Dynamic Server Version 12.10.FC8W2WE -- On-Line (Prim) -- Up 3 days 20:52:48 -- 1034088 Kbytes

However in the XI I am still seeing the following why is that? PLEASE SEE THE ATTACHMENT.


Something i notice. I moved the script over to the other servers change the name like you mentiond for the export INFORMIXSERVER= , but when doing su - nagios I am getting a weird result.

Code: Select all

[root@louie libexec]# su - nagios
Last login: Fri Jul 14 07:16:06 PDT 2017 on pts/3
su: warning: cannot change directory to /home/nagios: No such file or directory
-bash-4.2$

However when I cat /etc/passwd I see this --

Code: Select all

tpeters:x:1408:3001::/home/rmca73:/sbin/nologin
tkarwan:x:1409:3001::/home/rmca73:/sbin/nologin
carlosa:x:1410:3001::/home/rmca73:/sbin/nologin
nagios:x:224:499::/home/nagios:/bin/bash
[root@louie libexec]#

any idea in regards to this? right now when I run the script on the other servers I am seeing the same results. I see this.

Code: Select all

[root@louie libexec]#  /usr/local/nagios/libexec/check_ifx.sh
CRITICAL
[root@louie libexec]#

[dwhitfield]

Does running mkdir -p /home/nagios on the other servers resolve the issue?

It's not clear to me...is XI showing the same thing for all three servers?

[lmiltchev]

It's possible that someone in your company removed the nagios home directory. You will have to recreate it. You can also try adding the following lines in sudoers file on the client machine:

Code: Select all

nagios ALL=NOPASSWD: /usr/local/nagios/libexec/check_ifx.sh *
nagios ALL=NOPASSWD: /usr/informix/bin/onstat *

I didn't have to do this on my test VM, but this is a "vanilla" system - it hasn't been modified ("hardened") in any way.

Here's an example of a "working" setup.

On the client:

Env variables:

Code: Select all

[root@speedking ~]# env | grep -i informix
INFORMIXDIR=/opt/Informix_Software_Bundle
INFORMIXSERVER=speedking

Command definition:

Code: Select all

command[check_ifx]=/usr/local/nagios/libexec/check_ifx.sh $ARG1$

On the Nagios XI server

Service Definition:

Code: Select all

define service {
        host_name                       CentOS6-NRPE
        service_description             Informix
        use                             xiwizard_nrpe_service
        check_command                   check_nrpe!check_ifx!-a '/opt/Informix_Software_Bundle speedking'!!!!!!
        max_check_attempts              5
        check_interval                  5
        retry_interval                  1
        check_period                    xi_timeperiod_24x7
        notification_interval           60
        notification_period             xi_timeperiod_24x7
        notifications_enabled           1
        contacts                        nagiosadmin
        _xiwizard                       linux-server
        register                        1
        }

Testing from the CLI:

Code: Select all

# /usr/local/nagios/libexec/check_nrpe -H x.x.x.x -c check_ifx -a '/opt/Informix_Software_Bundle speedking'
IBM Informix Dynamic Server Version 12.10.FC8TL -- On-Line -- Up 20:17:13 -- 254088 Kbytes

In the CCM:

example01.PNG

In the GUI (Service Status Detail):

example02.PNG

Hope this helps.

[donnyforbes]

Does running mkdir -p /home/nagios on the other servers resolve the issue?

It's not clear to me...is XI showing the same thing for all three servers?

Yes XI is showing the same for all 3 servers. I ran this on the remote servers duey & louie

Code: Select all

mkdir -p /home/nagios

Code: Select all

[root@duey libexec]# su - nagios
Last login: Fri Jul 14 07:56:10 PDT 2017 on pts/0
-bash-4.2$ whoami
nagios
-bash-4.2$  /usr/local/nagios/libexec/check_ifx.sh
CRITICAL
-bash-4.2$

It doesn't return the same as the huey which is the first one we were working on.

[donnyforbes]

It's possible that someone in your company removed the nagios home directory. You will have to recreate it. You can also try adding the following lines in sudoers file on the client machine:

Code: Select all

nagios ALL=NOPASSWD: /usr/local/nagios/libexec/check_ifx.sh *
nagios ALL=NOPASSWD: /usr/informix/bin/onstat *

I didn't have to do this on my test VM, but this is a "vanilla" system - it hasn't been modified ("hardened") in any way.

Here's an example of a "working" setup.

On the client:

Env variables:

Code: Select all

[root@speedking ~]# env | grep -i informix
INFORMIXDIR=/opt/Informix_Software_Bundle
INFORMIXSERVER=speedking

Command definition:

Code: Select all

command[check_ifx]=/usr/local/nagios/libexec/check_ifx.sh $ARG1$

On the Nagios XI server

Service Definition:

Code: Select all

define service {
        host_name                       CentOS6-NRPE
        service_description             Informix
        use                             xiwizard_nrpe_service
        check_command                   check_nrpe!check_ifx!-a '/opt/Informix_Software_Bundle speedking'!!!!!!
        max_check_attempts              5
        check_interval                  5
        retry_interval                  1
        check_period                    xi_timeperiod_24x7
        notification_interval           60
        notification_period             xi_timeperiod_24x7
        notifications_enabled           1
        contacts                        nagiosadmin
        _xiwizard                       linux-server
        register                        1
        }

Testing from the CLI:

Code: Select all

# /usr/local/nagios/libexec/check_nrpe -H x.x.x.x -c check_ifx -a '/opt/Informix_Software_Bundle speedking'
IBM Informix Dynamic Server Version 12.10.FC8TL -- On-Line -- Up 20:17:13 -- 254088 Kbytes

In the CCM:

example01.PNG

In the GUI (Service Status Detail):

example02.PNG

Hope this helps.

No I am the only admin that works on these servers.

[donnyforbes]

I am a bit confused on what you want me to do. Can you please let me know. There seems to be a lot on here in regards to different people replying.. Please make it clear.

Thanks

[dwhitfield]

It's possible that someone in your company removed the nagios home directory. You will have to recreate it.

This is the same thing I said. I just gave you the command to do it.

If multiple people respond, answer the questions from both posts and run all of the suggested commands. Make sure you let us know if there's anything specific that isn't clear.

On the remote hosts (aka clients), add the following two lines to your /etc/sudoers:

Code: Select all

nagios ALL=NOPASSWD: /usr/local/nagios/libexec/check_ifx.sh *
nagios ALL=NOPASSWD: /usr/informix/bin/onstat *

After adding that, let us know if there are any changes.

[donnyforbes]

Please see below.

Code: Select all

[root@huey libexec]# env | grep -i informix
INFORMIXDIR=/usr/informix
PATH=/usr/informix/bin:/usr/informix/extend/krakatoa/jre/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
INFORMIXSERVER=hueynet
INFORMIXSQLHOSTS=/usr/informix/etc/sqlhosts
[root@huey libexec]#

Code: Select all

command[check_ifx]=/usr/lccal/nagios/libexec/check_ifx.sh $ARG1$ $ARG2$

On the Nagios XI server - Where is this and where do I run this from ?

Code: Select all

# /usr/local/nagios/libexec/check_nrpe -H x.x.x.x -c check_ifx -a '/opt/Informix_Software_Bundle speedking'

[donnyforbes]

It's possible that someone in your company removed the nagios home directory. You will have to recreate it.

This is the same thing I said. I just gave you the command to do it.

If multiple people respond, answer the questions from both posts and run all of the suggested commands. Make sure you let us know if there's anything specific that isn't clear.

On the remote hosts (aka clients), add the following two lines to your /etc/sudoers:

Code: Select all

nagios ALL=NOPASSWD: /usr/local/nagios/libexec/check_ifx.sh *
nagios ALL=NOPASSWD: /usr/informix/bin/onstat *

After adding that, let us know if there are any changes.

where in the file do I add these ??
nagios ALL=NOPASSWD: /usr/local/nagios/libexec/check_ifx.sh *
nagios ALL=NOPASSWD: /usr/informix/bin/onstat *

[donnyforbes]

I ran this from the XI server

Code: Select all

[root@nag libexec]# /usr/local/nagios/libexec/check_nrpe -H 192.168.1.42 -c check_ifx -a '/usr/informix hueynet'
NRPE: Unable to read output
[root@nag libexec]#

And this is the same error I see in the XI console. I also view the config on the XI console and posted it below.

Code: Select all

define service {
	host_name			huey.sg1.allamericanasphalt.com
	service_description		Informix
	use				xiwizard_nrpe_service
	display_name			Infx
	check_command			check_nrpe!check_ifx!-a '/usr/informix hueynet'!!!!!!
	initial_state			c
	max_check_attempts		5
	check_interval			5
	retry_interval			1
	check_period			xi_timeperiod_24x7
	notification_interval		60
	notification_period		xi_timeperiod_24x7
	notifications_enabled		1
	contacts			nagiosadmin
	register			1
	}	

where do I add this in the file what section?

Code: Select all

nagios ALL=NOPASSWD: /usr/local/nagios/libexec/check_ifx.sh *
nagios ALL=NOPASSWD: /usr/informix/bin/onstat *

Do I add it here..

Code: Select all

# NEEDED TO ALLOW NAGIOS TO CHECK SERVICE STATUS
Defaults:nagios !requiretty
nagios ALL=NOPASSWD: /usr/local/nagios/libexec/check_init_service

I checked the server that the script work on and it does not have these in the /etc/sudoers .... but maybe that's why its not showing on as OK in the XI

I added it here.

# NEEDED TO ALLOW NAGIOS TO CHECK SERVICE STATUS
Defaults:nagios !requiretty
nagios ALL=NOPASSWD: /usr/local/nagios/libexec/check_init_service
nagios ALL=NOPASSWD: /usr/local/nagios/libexec/check_ifx.sh *
nagios ALL=NOPASSWD: /usr/informix/bin/onstat *

I am still seeing this in the XI. What am I doing wrong??? Please advise.



Why when I run the following on the other two servers I see this for the output?

Code: Select all

[root@louie libexec]# /usr/local/nagios/libexec/check_ifx.sh
CRITICAL

When I run it on the main server the first one I see this.

Code: Select all

[root@huey libexec]# /usr/local/nagios/libexec/check_ifx.sh
IBM Informix Dynamic Server Version 12.10.FC8W2WE -- On-Line (Prim) -- Up 4 days 00:06:18 -- 1034088 Kbytes
[root@huey libexec]#

This should be the same on all servers unless I am missing something. I did modify the INFORMIXSERVER= (to the correct servers)
Now sure why this is happening any advise would be great.