Page 2 of 3

Re: Weird DNS issue, NRPE

Posted: Tue Aug 25, 2015 4:57 pm
by rkymtnhigh
When running as IP address this is the output of tcpdump:

Code: Select all

listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
15:51:11.751056 IP nagioshost.domain.com.42994 > monitoredhost.domain.com.nrpe: Flags [S], seq 1528144851, win 14600, options [mss 1460,sackOK,TS val 2989310634 ecr 0,nop,wscale 7], length 0
15:51:11.768523 IP monitoredhost.domain.com.nrpe > nagioshost.domain.com.42994: Flags [S.], seq 758733151, ack 1528144852, win 14480, options [mss 1380,sackOK,TS val 3127876922 ecr 2989310634,nop,wscale 7], length 0
15:51:11.768540 IP nagioshost.domain.com.42994 > monitoredhost.domain.com.nrpe: Flags [.], ack 1, win 115, options [nop,nop,TS val 2989310652 ecr 3127876922], length 0
15:51:11.768661 IP nagioshost.domain.com.42994 > monitoredhost.domain.com.nrpe: Flags [P.], seq 1:128, ack 1, win 115, options [nop,nop,TS val 2989310652 ecr 3127876922], length 127
15:51:11.780162 IP monitoredhost.domain.com.nrpe > nagioshost.domain.com.42994: Flags [.], ack 128, win 114, options [nop,nop,TS val 3127876940 ecr 2989310652], length 0
15:51:11.788132 IP monitoredhost.domain.com.nrpe > nagioshost.domain.com.42994: Flags [P.], seq 1:217, ack 128, win 114, options [nop,nop,TS val 3127876946 ecr 2989310652], length 216
15:51:11.788149 IP nagioshost.domain.com.42994 > monitoredhost.domain.com.nrpe: Flags [.], ack 217, win 123, options [nop,nop,TS val 2989310671 ecr 3127876946], length 0
15:51:11.788602 IP nagioshost.domain.com.42994 > monitoredhost.domain.com.nrpe: Flags [P.], seq 128:254, ack 217, win 123, options [nop,nop,TS val 2989310672 ecr 3127876946], length 126
15:51:11.805188 IP monitoredhost.domain.com.nrpe > nagioshost.domain.com.42994: Flags [P.], seq 217:443, ack 254, win 114, options [nop,nop,TS val 3127876960 ecr 2989310672], length 226
15:51:11.805764 IP nagioshost.domain.com.42994 > monitoredhost.domain.com.nrpe: Flags [P.], seq 254:1319, ack 443, win 131, options [nop,nop,TS val 2989310689 ecr 3127876960], length 1065
15:51:11.856934 IP monitoredhost.domain.com.nrpe > nagioshost.domain.com.42994: Flags [.], ack 1319, win 130, options [nop,nop,TS val 3127877017 ecr 2989310689], length 0
15:51:16.873885 IP monitoredhost.domain.com.nrpe > nagioshost.domain.com.42994: Flags [P.], seq 443:1508, ack 1319, win 130, options [nop,nop,TS val 3127882032 ecr 2989310689], length 1065
15:51:16.873944 IP nagioshost.domain.com.42994 > monitoredhost.domain.com.nrpe: Flags [P.], seq 1319:1350, ack 1508, win 148, options [nop,nop,TS val 2989315757 ecr 3127882032], length 31
15:51:16.873998 IP nagioshost.domain.com.42994 > monitoredhost.domain.com.nrpe: Flags [F.], seq 1350, ack 1508, win 148, options [nop,nop,TS val 2989315757 ecr 3127882032], length 0
15:51:16.890549 IP monitoredhost.domain.com.nrpe > nagioshost.domain.com.42994: Flags [P.], seq 1508:1539, ack 1350, win 130, options [nop,nop,TS val 3127882046 ecr 2989315757], length 31
15:51:16.890561 IP monitoredhost.domain.com.nrpe > nagioshost.domain.com.42994: Flags [F.], seq 1539, ack 1351, win 130, options [nop,nop,TS val 3127882046 ecr 2989315757], length 0
15:51:16.890567 IP nagioshost.domain.com.42994 > monitoredhost.domain.com.nrpe: Flags [.], ack 1540, win 148, options [nop,nop,TS val 2989315774 ecr 3127882046], length 0
And by hostname:

Code: Select all

listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
15:51:49.417290 IP nagioshost.domain.com.43090 > monitoredhost.domain.com.nrpe: Flags [S], seq 1284490412, win 14600, options [mss 1460,sackOK,TS val 2989348300 ecr 0,nop,wscale 7], length 0
15:51:49.429061 IP monitoredhost.domain.com.nrpe > nagioshost.domain.com.43090: Flags [S.], seq 1330691278, ack 1284490413, win 14480, options [mss 1380,sackOK,TS val 3127914589 ecr 2989348300,nop,wscale 7], length 0
15:51:49.429081 IP nagioshost.domain.com.43090 > monitoredhost.domain.com.nrpe: Flags [.], ack 1, win 115, options [nop,nop,TS val 2989348312 ecr 3127914589], length 0
15:51:49.429228 IP nagioshost.domain.com.43090 > monitoredhost.domain.com.nrpe: Flags [P.], seq 1:128, ack 1, win 115, options [nop,nop,TS val 2989348312 ecr 3127914589], length 127
15:51:49.443436 IP monitoredhost.domain.com.nrpe > nagioshost.domain.com.43090: Flags [.], ack 128, win 114, options [nop,nop,TS val 3127914601 ecr 2989348312], length 0
15:51:49.448192 IP monitoredhost.domain.com.nrpe > nagioshost.domain.com.43090: Flags [P.], seq 1:217, ack 128, win 114, options [nop,nop,TS val 3127914606 ecr 2989348312], length 216
15:51:49.448208 IP nagioshost.domain.com.43090 > monitoredhost.domain.com.nrpe: Flags [.], ack 217, win 123, options [nop,nop,TS val 2989348331 ecr 3127914606], length 0
15:51:49.448660 IP nagioshost.domain.com.43090 > monitoredhost.domain.com.nrpe: Flags [P.], seq 128:254, ack 217, win 123, options [nop,nop,TS val 2989348331 ecr 3127914606], length 126
15:51:49.467312 IP monitoredhost.domain.com.nrpe > nagioshost.domain.com.43090: Flags [P.], seq 217:443, ack 254, win 114, options [nop,nop,TS val 3127914626 ecr 2989348331], length 226
15:51:49.468103 IP nagioshost.domain.com.43090 > monitoredhost.domain.com.nrpe: Flags [P.], seq 254:1319, ack 443, win 131, options [nop,nop,TS val 2989348351 ecr 3127914626], length 1065
15:51:49.519999 IP monitoredhost.domain.com.nrpe > nagioshost.domain.com.43090: Flags [.], ack 1319, win 130, options [nop,nop,TS val 3127914680 ecr 2989348351], length 0
Thank you!!

Re: Weird DNS issue, NRPE

Posted: Tue Aug 25, 2015 5:02 pm
by jdalrymple
It's replying... that's weird.

Next stop nrpe log.

in nrpe.cfg:

Code: Select all

debug=1
Then if it's being called by inetd you're done. If daemonized restart it.

Run your check again then look for the error in /var/log/messages

Re: Weird DNS issue, NRPE

Posted: Wed Aug 26, 2015 11:58 am
by rkymtnhigh
nrpe must be called by inetd, when I try to 'service nrpe restart' i get unrecognized service.

When I run my check again after setting debug=1 I don't see any entries being generated in the /var/log/messages file.

Thank you

Re: Weird DNS issue, NRPE

Posted: Wed Aug 26, 2015 12:03 pm
by jdalrymple
All very bizarre.

Please post:

Code: Select all

ps -ef | grep nrpe
grep -v ^# nrpe.cfg | grep -v ^$

Re: Weird DNS issue, NRPE

Posted: Wed Aug 26, 2015 12:21 pm
by rkymtnhigh
Yeah, very strange. Anyways:

Code: Select all

ps -ef | grep nrpe
root     15108 27669  0 11:17 pts/1    00:00:00 grep nrpe

Code: Select all

grep -v ^# nrpe.cfg | grep -v ^$
grep: nrpe.cfg: No such file or directory
Thank you

Re: Weird DNS issue, NRPE

Posted: Wed Aug 26, 2015 12:25 pm
by jdalrymple
rkymtnhigh wrote:

Code: Select all

ps -ef | grep nrpe
root     15108 27669  0 11:17 pts/1    00:00:00 grep nrpe
This indicates that you're using xinetd to spawn nrpe
rkymtnhigh wrote:

Code: Select all

grep -v ^# nrpe.cfg | grep -v ^$
grep: nrpe.cfg: No such file or directory
Sorry, I'll need you to replace nrpe.cfg with /the/full/path/to/your/nrpe.cfg

I don't know where yours is.

Re: Weird DNS issue, NRPE

Posted: Wed Aug 26, 2015 12:52 pm
by rkymtnhigh
my apologies:

Code: Select all

grep -v ^# /usr/local/nagios/etc/nrpe.cfg | grep -v ^$
log_facility=daemon
pid_file=/var/run/nrpe.pid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
allowed_hosts=127.0.0.1

dont_blame_nrpe=1
allow_bash_command_substitution=0
debug=1
command_timeout=60
connection_timeout=300
command[check_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10
command[check_load]=/usr/local/nagios/libexec/check_load -w 15,10,5 -c 30,25,20
command[check_hda1]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/hda1
command[check_zombie_procs]=/usr/local/nagios/libexec/check_procs -w 5 -c 10 -s Z
command[check_total_procs]=/usr/local/nagios/libexec/check_procs -w 150 -c 200
command[check_mem]=/usr/lib64/nagios/plugins/check_mem  -f -w 15 -c 10

Re: Weird DNS issue, NRPE

Posted: Wed Aug 26, 2015 4:41 pm
by jdalrymple
Here is the only thing I can think of at this point, are we looking at the Nagios server or the monitored server?
I suppose I should point out to you that nrpe.cfg is irrelevant on a Nagios server, only the servers being monitored? This may be why you're not seeing any debug messages.

So on the monitored server (not Nagios server) please put the debug=1 line into nrpe.cfg
restart nrpe (if it's daemonized)
run your check again
`grep nrpe /var/log/messages` - again on the monitored server

If you already are working on the monitored server - well then I'm lost. No clue what could be preventing logging from happening.

Re: Weird DNS issue, NRPE

Posted: Thu Aug 27, 2015 12:47 pm
by rkymtnhigh
I apologize, I am still fairly new at linux and Nagios. I was doing this on the Nagios host nrpe.cfg file.
Once I configured the monitored server's nrpe.cfg file to debug=1, I was able to get some entries.

Here is what I logged in /var/log/messages | grep nrpe while I ran my unsuccessfully check:

Code: Select all

Aug 27 11:42:47 monitoredhost xinetd[1983]: EXIT: nrpe status=0 pid=18822 duration=5(sec)
Aug 27 11:42:50 monitoredhost xinetd[1983]: START: nrpe pid=18852 from=::ffff:192.168.XXX.XXX
Aug 27 11:42:50 monitoredhost nrpe[18852]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Aug 27 11:42:50 monitoredhost xinetd[1983]: EXIT: nrpe status=0 pid=18852 duration=0(sec)
Aug 27 11:42:50 monitoredhost xinetd[1983]: START: nrpe pid=18855 from=::ffff:192.168.XXX.XXX
Aug 27 11:42:50 monitoredhost nrpe[18855]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Aug 27 11:42:50 monitoredhost xinetd[1983]: EXIT: nrpe status=0 pid=18855 duration=0(sec)
Aug 27 11:42:53 monitoredhost xinetd[1983]: START: nrpe pid=18858 from=::ffff:192.168.XXX.XXX
Aug 27 11:42:53 monitoredhost nrpe[18858]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Aug 27 11:42:53 monitoredhost xinetd[1983]: EXIT: nrpe status=0 pid=18858 duration=0(sec)
Aug 27 11:42:57 monitoredhost xinetd[1983]: START: nrpe pid=18881 from=::ffff:172.29.XXX.XXX
Aug 27 11:42:57 monitoredhost nrpe[18881]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Aug 27 11:42:59 monitoredhost xinetd[1983]: START: nrpe pid=18892 from=::ffff:192.168.XXX.XXX
Aug 27 11:42:59 monitoredhost nrpe[18892]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Aug 27 11:42:59 monitoredhost xinetd[1983]: EXIT: nrpe status=0 pid=18892 duration=0(sec)
Aug 27 11:43:02 monitoredhost xinetd[1983]: EXIT: nrpe status=0 pid=18881 duration=5(sec)
Aug 27 11:43:07 monitoredhost xinetd[1983]: START: nrpe pid=18923 from=::ffff:172.29.XXX.XXX
Aug 27 11:43:07 monitoredhost nrpe[18923]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Aug 27 11:43:07 monitoredhost xinetd[1983]: EXIT: nrpe status=0 pid=18923 duration=0(sec)
Aug 27 11:43:08 monitoredhost xinetd[1983]: START: nrpe pid=18926 from=::ffff:192.168.XXX.XXX
Aug 27 11:43:08 monitoredhost nrpe[18926]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Aug 27 11:43:08 monitoredhost xinetd[1983]: EXIT: nrpe status=0 pid=18926 duration=0(sec)

Re: Weird DNS issue, NRPE

Posted: Thu Aug 27, 2015 12:53 pm
by jdalrymple
It appears you have AAAA records, but perhaps not the proper IPv6 PTR records. Can you verify?

Maybe just disabling IPv6 would be the proper solution.