Outbound and Inbound Xfer Questions

[abrist]

Is this this the only host having this problem?
Are any of the checks unique to it?

What's with the 0

Not sure. Lets check the unconfigured objects log:

Code: Select all

cat /usr/local/nagiosxi/var/corelog.*

[BanditBBS]

All hosts are having issues from both servers. I'm going to do another tcpdump on both servers, initiate checks and then look at the dumps with wireshark. Something is really not working as it should.

Here is the data you wanted:

Code: Select all

4087422
[1383173633] Warning:  Passive check result was received for service 'Cron Scheduling Daemon' on host 'rp000001', but the service could not be found!
[1383173633] Warning:  Passive check result was received for service 'CPU Stats' on host 'rp000001', but the service could not be found!
[1383173633] Warning:  Passive check result was received for service 'Cron Scheduling Daemon' on host 'rp000001', but the service could not be found!
[1383173633] Warning:  Passive check result was received for service 'CPU Stats' on host 'rp000001', but the service could not be found!
[1383173650] SERVICE ALERT: svwdccpcm03;SNMP Traps;WARNING;HARD;1;This Notification indicates that at least one gateway has attempted to register or communicate with the CallManager and failed. 4 AN1930D677B2413 1 10.95.255.252 3 / enterprises.9.9.156.1.10.1 ():4 enterprises.9.9.156.1.3.1.1.2 ():AN1930D677B2413 enterprises.9.9.156.1.3.1.1.7 ():1 enterprises.9.9.156.1.3.1.1.8 ():10.95.255.252 enterprises.9.9.156.1.10.28 ():3
[1383173650] SERVICE ALERT: svwdccpcm03;SNMP Traps;WARNING;HARD;1;This Notification indicates that at least one gateway has attempted to register or communicate with the CallManager and failed. 4 AN1930D677B2413 1 10.95.255.252 3 / enterprises.9.9.156.1.10.1 ():4 enterprises.9.9.156.1.3.1.1.2 ():AN1930D677B2413 enterprises.9.9.156.1.3.1.1.7 ():1 enterprises.9.9.156.1.3.1.1.8 ():10.95.255.252 enterprises.9.9.156.1.10.5 ():3
a:2:{s:8:"rp000001";a:2:{s:9:"last_seen";i:1383173651;s:8:"services";a:10:{s:22:"Cron Scheduling Daemon";i:1383173651;s:9:"CPU Stats";i:1383173651;s:16:"/boot Disk Usage";i:1383173591;s:14:"/MQHA/FFOMSP02";i:1383172991;s:12:"/ Disk Usage";i:1383172391;s:14:"/MQHA/FFCLSP02";i:1383172091;s:14:"/MQHA/FFOMSP01";i:1383170591;s:14:"/MQHA/FFCLSP01";i:1383170291;s:13:"Dummy Service";i:1383169391;s:10:"Swap Usage";i:1383166991;}}s:8:"rp000002";a:4:{s:9:"last_seen";i:1383173231;s:8:"services";a:1:{i:0;i:1383173231;}s:15:"hidden_services";N;s:8:"hide_all";b:0;}}

Here it is on another server. If I go to unconfigured objects NOTHING is listed on this server except staging.aemq01

Code: Select all

[clarkj@svwddnagios01 ~]$ cat /usr/local/nagiosxi/var/corelog.*
2603672
[1383173400] SERVICE ALERT: aewdapp03;11commerce_sessions;CRITICAL;HARD;2;CRITICAL: Too many sessions  - 2266
a:10:{s:9:"dev.app02";a:2:{s:9:"last_seen";i:1383173231;s:8:"services";a:1:{i:0;i:1383173231;}}s:8:"aewdf501";a:4:{s:9:"last_seen";i:1383173052;s:8:"services";a:1:{i:0;i:1383173052;}s:15:"hidden_services";N;s:8:"hide_all";b:0;}s:7:"dev.svn";a:4:{s:9:"last_seen";i:1383172931;s:8:"services";a:1:{i:0;i:1383172931;}s:15:"hidden_services";N;s:8:"hide_all";b:0;}s:11:"dev.qaapp01";a:4:{s:9:"last_seen";i:1383172631;s:8:"services";a:1:{i:0;i:1383172631;}s:15:"hidden_services";N;s:8:"hide_all";b:0;}s:8:"dev.db01";a:4:{s:9:"last_seen";i:1383172332;s:8:"services";a:1:{i:0;i:1383172332;}s:15:"hidden_services";N;s:8:"hide_all";b:0;}s:8:"aewdf502";a:4:{s:9:"last_seen";i:1383168132;s:8:"services";a:1:{i:0;i:1383168132;}s:15:"hidden_services";N;s:8:"hide_all";b:0;}s:10:"dev.diablo";a:4:{s:9:"last_seen";i:1383163091;s:8:"services";a:1:{i:0;i:1383163091;}s:15:"hidden_services";N;s:8:"hide_all";b:0;}s:9:"dev.app01";a:4:{s:9:"last_seen";i:1383136451;s:8:"services";a:1:{i:0;i:1383136451;}s:15:"hidden_services";N;s:8:"hide_all";b:0;}s:14:"staging.aemq01";a:4:{s:9:"last_seen";i:1383125651;s:8:"services";a:2:{i:0;i:1383125651;i:1;i:1383124931;}s:15:"hidden_services";N;s:8:"hide_all";b:0;}s:9:"dev.web01";a:4:{s:9:"last_seen";i:1383049452;s:8:"services";a:1:{i:0;i:1383049452;}s:15:"hidden_services";N;s:8:"hide_all";b:0;}}

[BanditBBS]

So, I tried all kind of stuff this evening, can't find any reason. I did two pcaps from the 2 servers, but couldnt figure it out in wireshark. The data just seems to be garbled in it to me.

I noticed doubled up results on the /boot partition check, here is the command run at the cli. In XI on the receiving server it should the results, plus the results of one of the other disk checks as well

Code: Select all

[clarkj@svwddnagios01 libexec]$ ./check_nrpe -H rp000001.aeo.ae.com -c check_disk -a '-w 20% -c 10% -p /boot'
DISK OK - free space: /boot 371 MB (80% inode=99%);| /boot=88MB;387;435;0;484

EDIT #1:Should I just change to NRDP? And before you say yes, make sure I can choose a custom port I dont want to ask yet again for more ports. The network team has been bending over backwards for me. I'd truly like to just figure out this issue!

EDIT #2: Ports were opened one direction already and tested...NRDP works fine, it is an issue with NSCA. Going to try and get port 443 open the other direction. Any reason to not use NRDP and keep trying NSCA?

EDIT #3: I'd love to figure out the NSCA issue and not holding my breath that 443 will get opened tomorrow. I'm getting drunk and going to bed, 12 hour days are killing me!

EDIT #4: I got the port opened, so I can do NRDP on both now. If you'd like me to test anything for your curiosity, throw it out there today, as after today I 'm moving to NRDP and forgetting NSCA

[abrist]

What versions of nsca are you running? Some of the entries from those unconfigured object logs do not look right. You may be on to something here. Once you report the versions I will try to reproduce the issue in a test environment.

[BanditBBS]

What versions of nsca are you running? Some of the entries from those unconfigured object logs do not look right. You may be on to something here. Once you report the versions I will try to reproduce the issue in a test environment.

help me out here Andy (ouch, that hurt typing that),

How do I find the version? I know the one server is whatever XI 2012R2.5 would install. The other server was originally installed just about a year ago, and is currently running XI 2012R2.3. I just don't know how to find the NSCA version.

[tmcdonald]

/usr/local/nagios/bin/nsca | grep Version

Or whatever location "find / -name nsca" tells you.

[BanditBBS]

Both servers are Version: 2.9.1

[BanditBBS]

Both servers are now using NRDP and I couldn't be happier. Just wish I knew where the issue was, XI sending the packets or XI receiving the packets...but eh, I dont care now....lock it up if you'd like.

[abrist]

I will leave this topic open for now, just in case this is a bug I would like to see if anyone necros this thread over the next week or two.

[BanditBBS]

Andy,

i just thought of something. I have been using a send_nsca.exe to send items from SCOM into one of my XI servers and it works great. So receiving NSCA works fine, it has to be XI sending out NSCA where the issue is.