Page 3 of 4
Re: Monitor Isilon with Nagios XI
Posted: Tue Aug 11, 2015 8:52 am
by SilvesterDU
I did before my fist try. Anyway I tried your suggestion:
1. Removed the line and re-run. No error and no warning. This is expected. As there is no reference to my isilon-services.cfg.
2. Add the line back and re-run. Same error.
I'm considering re-deploying an Nagios XI instance, if it is the last resort before I gave up...
Silvester
Re: Monitor Isilon with Nagios XI
Posted: Tue Aug 11, 2015 10:35 am
by jdalrymple
SilvesterDU wrote:As there is no reference to my isilon-services.cfg.
That is not correct. In your nagios.cfg:
Code: Select all
# STATIC OBJECT DEFINITIONS (THESE DON'T GET EXPORTED/IMPORTED BY NAGIOSQL)
cfg_dir=/usr/local/nagios/etc/static
When you restarted and got no error you should have looked in your interface. It's likely you didn't get an error because things simply were working. This is why I had you remove the line that effectively duplicated the host.
Re: Monitor Isilon with Nagios XI
Posted: Tue Aug 11, 2015 11:21 am
by SilvesterDU
jdalrymple wrote:SilvesterDU wrote:As there is no reference to my isilon-services.cfg.
That is not correct. In your nagios.cfg:
Code: Select all
# STATIC OBJECT DEFINITIONS (THESE DON'T GET EXPORTED/IMPORTED BY NAGIOSQL)
cfg_dir=/usr/local/nagios/etc/static
When you restarted and got no error you should have looked in your interface. It's likely you didn't get an error because things simply were working. This is why I had you remove the line that effectively duplicated the host.
OK, you're right. BTW, the cfg_dir line is born with nagios.cfg. So the lesson here is we don't need to add cfg_file in the nagios.cfg if the config file is located in the static folder.
At least we are making progress. Now the error is host not reachable.
Nagios_networkstatus.png
In my environment, the Nagios appliance is not located in the same network as Isilon, so I use another machine as gateway. The pic below shows I can ping Isilon or traceroute from within the Nagios appliance through another hop.
Nagios_networkstatus_ping.png
Nagios_tracerout.png
Any suggestions how to resolve this host unreachable issue?
Thanks a lot!
Silvester
Re: Monitor Isilon with Nagios XI
Posted: Tue Aug 11, 2015 1:18 pm
by jolson
One option is to add a static route to your Nagios box so that it's aware of the Isilon network at all times (as opposed to needing redirection). I assume the ping command can not handle the redirection taking place properly.
Code: Select all
route add -net 10.6.120.0 netmask 255.255.255.0 gw 192.168.102.101 dev eth0
Does the above work properly? You simply need to tell Nagios which gateway to use in the case of attempted access to the 10.6.120.0/24 network. I am almost certain that the gateway I have listed in the above command is incorrect, so be sure to use the proper gateway there. I am also assuming that your 10.6.120.163 network is using a /24 subnet mask, so make changes as necessary to the subnet mask as well. In addition to those assumptions, I am guessing that your primary interface is 'eth0' - be sure to verify that as well.
To set this route persistently (across reboots), you will need to edit your network interface script:
Code: Select all
vi /etc/sysconfig/network-scripts/route-eth0
Add:
GATEWAY0=192.168.102.101
NETMASK0=255.255.255.0
ADDRESS0=10.6.120.0
Once you are capable of reaching the Isilon device without requiring a redirection - let us know. Be sure to be careful with these settings, as I've made a lot of assumptions above (gateway, netmask, device, etc).
Re: Monitor Isilon with Nagios XI
Posted: Wed Aug 12, 2015 4:17 am
by SilvesterDU
Thank you for the hint-- actually I takes a safer way: instead of modifying the network setting of Nagios appliance, I added an network interface on my Isilon nodes and ensure that interface is in the same subnet of Nagios.
Now the host is shown as "up".
Nagios_hostup2.png
Then I noticed all the services is showing "Nagios has not received an update from rtpsolisilon-1 for at least 25 hours".
Nagios_serviceDetails_issue.png
Also when I checked the "isilon-checks.log" (Isilon check log from /libexec/ce/isilon-checks.sh), it reports "Remote command execution failed: Host key verification failed."
Could these 2 issues be related? I attached the isilon-checks.sh script as well.
You have been very helpful, thanks again.
Silvester
Re: Monitor Isilon with Nagios XI
Posted: Wed Aug 12, 2015 4:20 pm
by ssax
Try running these commands to save the SSH key:
Code: Select all
su - nagios
ssh user@isilonaddress
Then accept the SSH key and exit and try again.
Re: Monitor Isilon with Nagios XI
Posted: Wed Aug 12, 2015 10:16 pm
by SilvesterDU
I've tried but seems the same result:(
Silvester
Re: Monitor Isilon with Nagios XI
Posted: Thu Aug 13, 2015 12:08 pm
by jdalrymple
Whose cron did you add this to?
Code: Select all
*/5 * * * * /opt/nagios/libexec/ce/isilon-checks.sh > /opt/nagios/var/isilon-checks.log
Whoever that is will need to be able to ssh to the isilon without any keyboard interaction I suspect. That's the debugging step I'd take next.
Re: Monitor Isilon with Nagios XI
Posted: Thu Aug 13, 2015 10:24 pm
by SilvesterDU
jdalrymple wrote:Whose cron did you add this to?
Code: Select all
*/5 * * * * /opt/nagios/libexec/ce/isilon-checks.sh > /opt/nagios/var/isilon-checks.log
Whoever that is will need to be able to ssh to the isilon without any keyboard interaction I suspect. That's the debugging step I'd take next.
I add to the cron of root user, and root can ssh to the Isilon without keyboard interaction.
So, we need to add the account "nagios" to Isilon as well so that nagios can ssh to Isilon without any password/pass-phrase?
I will have a try and let you know the result
Thanks
Silvester
Re: Monitor Isilon with Nagios XI
Posted: Fri Aug 14, 2015 1:08 pm
by ssax
Sounds good, let us know the results.