There's exactly one xinetd running on the remote host:
Code: Select all
[root@vbgmadb11 nagios]# ps -aef|grep xinetd|grep -v grep
root 947 1 0 Dec11 ? 00:00:00 xinetd -stayalive -pidfile /var/run/xinetd.pid
[root@vbgmadb11 nagios]# xinetd -version
xinetd Version 2.3.14 libwrap loadavg
So I stop nagios:
Code: Select all
[root@vbgmadb11 etc]# ps -aef|grep nagios|grep -v grep
nagios 27811 1 13:19 0:00 0.1 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
madpbk51:/usr/local/nagios/etc # cd servers/
madpbk51:/usr/local/nagios/etc/servers # /etc/init.d/nagios stop
madpbk51:/usr/local/nagios/etc/servers # ps -aef|grep nagios
root 1468 30617 0 14:22 pts/0 00:00:00 grep nagios
I stop xinetd on the remote host:
Code: Select all
[root@vbgmadb11 etc]# service xinetd stop
Stopping xinetd: [ OK ]
[root@vbgmadb11 etc]# tail /var/log/debug
...
Dec 18 14:15:56 vbgmadb11 xinetd[17219]: Exiting...
[root@vbgmadb11 etc]# ps -aef|grep xinetd
root 24477 22974 0 14:18 pts/0 00:00:00 grep xinetd
Now I edit services.cfg on the nagios server to check check_lsnr on the remote machine vbmadb11.
Then I start xinetd on the remote host.
Code: Select all
[root@vbgmadb11 etc]# service xinetd start
Starting xinetd: [ OK ]
[root@vbgmadb11 etc]# ps -aef|grep xinetd|grep -v grep
root 25407 1 0 14:26 ? 00:00:00 xinetd -stayalive -pidfile /var/run/xinetd.pid
Output of /var/log/debug of the remote host:
Code: Select all
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Reading included configuration file: /etc/xinetd.d/chargen-dgram [file=/etc/xinetd.conf] [line=49]
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Reading included configuration file: /etc/xinetd.d/chargen-stream [file=/etc/xinetd.d/chargen-stream] [line=67]
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Reading included configuration file: /etc/xinetd.d/daytime-dgram [file=/etc/xinetd.d/daytime-dgram] [line=67]
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Reading included configuration file: /etc/xinetd.d/daytime-stream [file=/etc/xinetd.d/daytime-stream] [line=67]
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Reading included configuration file: /etc/xinetd.d/discard-dgram [file=/etc/xinetd.d/discard-dgram] [line=67]
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Reading included configuration file: /etc/xinetd.d/discard-stream [file=/etc/xinetd.d/discard-stream] [line=67]
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Reading included configuration file: /etc/xinetd.d/echo-dgram [file=/etc/xinetd.d/echo-dgram] [line=67]
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Reading included configuration file: /etc/xinetd.d/echo-stream [file=/etc/xinetd.d/echo-stream] [line=67]
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Reading included configuration file: /etc/xinetd.d/eklogin [file=/etc/xinetd.d/eklogin] [line=67]
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Reading included configuration file: /etc/xinetd.d/ekrb5-telnet [file=/etc/xinetd.d/ekrb5-telnet] [line=13]
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Reading included configuration file: /etc/xinetd.d/gssftp [file=/etc/xinetd.d/gssftp] [line=14]
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Reading included configuration file: /etc/xinetd.d/klogin [file=/etc/xinetd.d/klogin] [line=14]
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Reading included configuration file: /etc/xinetd.d/krb5-telnet [file=/etc/xinetd.d/krb5-telnet] [line=13]
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Reading included configuration file: /etc/xinetd.d/kshell [file=/etc/xinetd.d/kshell] [line=13]
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Reading included configuration file: /etc/xinetd.d/nrpe [file=/etc/xinetd.d/nrpe] [line=13]
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Reading included configuration file: /etc/xinetd.d/rmcp [file=/etc/xinetd.d/rmcp] [line=16]
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Reading included configuration file: /etc/xinetd.d/rsync [file=/etc/xinetd.d/rsync] [line=34]
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Reading included configuration file: /etc/xinetd.d/tcpmux-server [file=/etc/xinetd.d/tcpmux-server] [line=13]
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Reading included configuration file: /etc/xinetd.d/tftp [file=/etc/xinetd.d/tftp] [line=68]
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Reading included configuration file: /etc/xinetd.d/time-dgram [file=/etc/xinetd.d/time-dgram] [line=18]
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Reading included configuration file: /etc/xinetd.d/time-stream [file=/etc/xinetd.d/time-stream] [line=67]
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: removing chargen
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: removing chargen
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: removing daytime
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: removing daytime
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: removing discard
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: removing discard
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: removing echo
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: removing echo
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: removing eklogin
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: removing telnet
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: removing ftp
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: removing klogin
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: removing telnet
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: removing kshell
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: removing asf-rmcp
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: removing asf-secure-rmcp
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: removing rsync
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: removing tcpmux
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: removing tftp
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: removing time
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: removing time
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: xinetd Version 2.3.14 started with libwrap loadavg labeled-networking options compiled in.
Dec 18 14:26:44 vbgmadb11 xinetd[25407]: Started working: 1 available service
Now I start nagios again.
Code: Select all
madpbk51:/usr/local/nagios/etc/servers # /etc/init.d/nagios start
Starting nagios: done.
I log in as user nagios on the nagios server:
Code: Select all
nagios@madpbk51:~> /usr/local/nagios/libexec/check_nrpe -H vbgmadb11 -c check_lsnr
Oracle Listener is working fine.
I have a look at /var/log/debug (on the remote host) where I can find that check_lsnr was called and executed:
Code: Select all
Dec 18 14:31:25 vbgmadb11 xinetd[25407]: START: nrpe pid=25880 from=194.59.103.56
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_hda1]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/xvda3
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_var]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /var
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_opt_oracle]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /opt/oracle
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_qc_fs]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /oradata/qcprod
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_qc_archive_log]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /oradata/qcprod/archive_log
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_qc_online_redo]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /oradata/qcprod/online_redo
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_aris_fs]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /oradata/arissvr/dbf
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_aris_archive_log]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /oradata/arissvr/archive_log
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_aris_online_redo]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /oradata/arissvr/online_redo
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_arisdb_fs]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /oradata/arispub/dbf
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_arisdb_archive_log]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /oradata/arispub/archive_log
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_arisdb_online_redo]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /oradata/arispub/online_redo
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_sfkarte_fs]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /oradata/sfkarte
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_kfsdb_fs]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /oradata/kfsdb01/db
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_kfsdb_archive_log]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /oradata/kfsdb01/archive
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_kfsdb_online_redo]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /oradata/kfsdb01/redo
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_dd_fs]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /oradata/ddprod/dbf
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_dd_archive_log]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /oradata/ddprod/archive
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_dd_online_redo1]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /oradata/ddprod/online01
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_dd_online_redo2]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /oradata/ddprod/online02
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_eth0]=/usr/local/nagios/libexec/check_network_device.bsh -E 0
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_messages_err]=/usr/local/nagios/libexec/check_log -F /var/log/messages -O /usr/local/nagios/libexec/messages.err -q "error"
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_messages_prob]=/usr/local/nagios/libexec/check_log -F /var/log/messages -O /usr/local/nagios/libexec/messages.problem -q "problem"
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_messages_fail]=/usr/local/nagios/libexec/check_log -F /var/log/messages -O /usr/local/nagios/libexec/messages.fail -q "fail"
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_messages_down]=/usr/local/nagios/libexec/check_log -F /var/log/messages -O /usr/local/nagios/libexec/messages.down -q "down"
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_qc]=/usr/local/nagios/libexec/check_oracle.bsh --db qcprod
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_QC_alert]=/usr/local/nagios/libexec/check_alertlog -F /opt/oracle/base/diag/rdbms/qcprod/qcprod/trace/alert_qcprod.log -O /usr/local/nagios/libexec/alert_QC.log
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_QC_alert_cp]=/usr/local/nagios/libexec/check_alertlog_checkpoint -F /opt/oracle/base/diag/rdbms/qcprod/qcprod/trace/alert_qcprod.log -O /usr/local/nagios/libexec/alert_QC_cp.log
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_ARIS]=/usr/local/nagios/libexec/check_oracle.bsh --db ARIS
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_ARIS_alert]=/usr/local/nagios/libexec/check_alertlog -F /opt/oracle/base/diag/rdbms/aris/ARIS/trace/alert_ARIS.log -O /usr/local/nagios/libexec/alert_ARIS.log
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_ARIS_alert_cp]=/usr/local/nagios/libexec/check_alertlog_checkpoint -F /opt/oracle/base/diag/rdbms/aris/ARIS/trace/alert_ARIS.log -O /usr/local/nagios/libexec/alert_ARIS_cp.log
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_ARISBP]=/usr/local/nagios/libexec/check_oracle.bsh --db ARISBP
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_ARISBP_alert]=/usr/local/nagios/libexec/check_alertlog -F /opt/oracle/base/diag/rdbms/arisbp/ARISBP/trace/alert_ARISBP.log -O /usr/local/nagios/libexec/alert_ARISBP.log
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_ARISBP_alert_cp]=/usr/local/nagios/libexec/check_alertlog_checkpoint -F /opt/oracle/base/diag/rdbms/arisbp/ARISBP/trace/alert_ARISBP.log -O /usr/local/nagios/libexec/alert_ARISBP_cp.log
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_sfkarte]=/usr/local/nagios/libexec/check_oracle.bsh --db sfkarteprod
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_sfk_alert]=/usr/local/nagios/libexec/check_alertlog -F /opt/oracle/base/diag/rdbms/sfkarteprod/sfkarteprod/trace/alert_sfkarteprod.log -O /usr/local/nagios/libexec/alert_sfk.log
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_sfk_alert_cp]=/usr/local/nagios/libexec/check_alertlog_checkpoint -F /opt/oracle/base/diag/rdbms/sfkarteprod/sfkarteprod/trace/alert_sfkarteprod.log -O /usr/local/nagios/libexec/alert_sfk_cp.log
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_dd]=/usr/local/nagios/libexec/check_oracle.bsh --db ddprod
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_dd_alert]=/usr/local/nagios/libexec/check_alertlog -F /opt/oracle/base/diag/rdbms/ddprod/ddprod/trace/alert_ddprod.log -O /usr/local/nagios/libexec/alert_dd.log
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_dd_alert_cp]=/usr/local/nagios/libexec/check_alertlog_checkpoint -F /opt/oracle/base/diag/rdbms/ddprod/ddprod/trace/alert_ddprod.log -O /usr/local/nagios/libexec/alert_dd_cp.log
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_kfsdb]=/usr/local/nagios/libexec/check_oracle.bsh --db KFSDB01
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_kfs_alert]=/usr/local/nagios/libexec/check_alertlog -F /opt/oracle/base/diag/rdbms/kfsdb01/KFSDB01/trace/alert_KFSDB01.log -O /usr/local/nagios/libexec/alert_kfs.log
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_kfs_alert_cp]=/usr/local/nagios/libexec/check_alertlog_checkpoint -F /opt/oracle/base/diag/rdbms/kfsdb01/KFSDB01/trace/alert_KFSDB01.log -O /usr/local/nagios/libexec/alert_kfs_cp.log
[b][color=#0000FF]Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_lsnr]=/usr/local/nagios/libexec/check_listener.bsh[/color][/b]
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_load2]=/usr/local/nagios/libexec/check_load -w 15,10,5 -c 30,25,20 -r
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_zombie_procs]=/usr/local/nagios/libexec/check_procs -w 5 -c 10 -s Z
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_total_procs]=/usr/local/nagios/libexec/check_procs -w 200 -c 250
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_time]=/usr/local/nagios/libexec/check_ntp_time -H ntp.dpma.de -w ~0.4:0.4 -c ~0.5:0.5
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_cores]=/usr/local/nagios/libexec/check_cores.bsh
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Added command[check_swap]=/usr/local/nagios/libexec/check_swap -w 20% -c 10%
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Handling the connection...
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Host is asking for command 'check_lsnr' to be run...
Dec 18 14:31:25 vbgmadb11 nrpe[25880]: Running command: /usr/local/nagios/libexec/check_listener.bsh
Dec 18 14:31:26 vbgmadb11 nrpe[25880]: Command completed with return code 0 and output: Oracle Listener is working fine.
Dec 18 14:31:26 vbgmadb11 nrpe[25880]: Return Code: 0, Output: Oracle Listener is working fine.
Dec 18 14:31:26 vbgmadb11 xinetd[25407]: EXIT: nrpe status=0 pid=25880 duration=1(sec)
I even can see the the command check_lsnr was loaded!
A few minutes later in the nagios server's /usr/local/nagios/var/nagios.log:
Code: Select all
...
[1387373516] SERVICE ALERT: vbgmadb11;Check Listener;CRITICAL;SOFT;1;NRPE: Command 'check_lsnr' not defined
[1387373636] SERVICE ALERT: vbgmadb11;Check Listener;CRITICAL;SOFT;2;NRPE: Command 'check_lsnr' not defined
[1387373756] SERVICE ALERT: vbgmadb11;Check Listener;CRITICAL;HARD;3;NRPE: Command 'check_lsnr' not defined
[1387373756] SERVICE NOTIFICATION: nagiosadmin;vbgmadb11;Check Listener;CRITICAL;notify-service-by-email;NRPE: Command check_lsnr not defined
...
I don't know what to say. Can you see anything interesting in the loggings?