Page 3 of 6
Re: All Linux Server CPU Spike at same time
Posted: Tue Mar 14, 2017 7:06 pm
by kwhogster
on the remote server
/etc/rsyslogd.conf
not found
wrong name or wrong folder?
Re: All Linux Server CPU Spike at same time
Posted: Tue Mar 14, 2017 8:04 pm
by dwhitfield
If we ever give you a command and the default path is not found, please run find / -name $nameoffile. Probably you can just edit whatever file it finds, but if it finds more than one or nothing, definitely let us know. There is a caveat to the more than one. If there are two and one of them is in the directory where you extracted Core, then that one can be ignored.
In this case, what does find / -name rsyslogd.conf return?
Re: All Linux Server CPU Spike at same time
Posted: Tue Mar 14, 2017 8:19 pm
by kwhogster
[root@tgcs018 /]# find / -name rsyslogd.conf
[root@tgcs018 /]#
nothing found
Re: All Linux Server CPU Spike at same time
Posted: Tue Mar 14, 2017 8:34 pm
by dwhitfield
It's probably /etc/syslogd.conf then, but if not, let us know if that find command finds it. It's not a typo. rsyslog is a different program than syslog.
Re: All Linux Server CPU Spike at same time
Posted: Tue Mar 14, 2017 8:47 pm
by kwhogster
I found this
/etc/rsyslog.conf
Re: All Linux Server CPU Spike at same time
Posted: Tue Mar 14, 2017 8:56 pm
by kwhogster
Ok got the messages log now
ran this on my Nagios server
root@tgcs017:~# /usr/local/nagios/libexec/check_nrpe -H 10.2.8.74
NRPE v2.15
Log from the remote server
Code: Select all
[root@tgcs018 /]# tail -n 100 /var/log/messages
Mar 14 19:51:54 tgcs018 xinetd[20332]: START: nrpe pid=6604 from=::ffff:10.2.8.79
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_asterisk]=/usr/local/nagios/libexec/check_asterisk.pl $ARG1$
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_sip]=/usr/local/nagios/libexec/check_sip $ARG1$
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_asterisk_sip_peers]=sudo /usr/local/nagios/libexec/check_asterisk_sip_peers.sh $ARG1$
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_asterisk_version]=/usr/local/nagios/libexec/nagisk.pl -c version
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_asterisk_peers]=/usr/local/nagios/libexec/nagisk.pl -c peers
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_asterisk_channels]=/usr/local/nagios/libexec/nagisk.pl -c channels
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_asterisk_zaptel]=/usr/local/nagios/libexec/nagisk.pl -c zaptel
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_asterisk_span]=/usr/local/nagios/libexec/nagisk.pl -c span -s 1
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_init_service]=sudo /usr/local/nagios/libexec/check_init_service $ARG1$
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_services]=/usr/local/nagios/libexec/check_services -p $ARG1$
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_users]=/usr/local/nagios/libexec/check_users $ARG1$
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_load]=/usr/local/nagios/libexec/check_load $ARG1$
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_swap]=/usr/local/nagios/libexec/check_swap $ARG1$
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_cpu_stats]=/usr/local/nagios/libexec/check_cpu_stats.sh $ARG1$
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_mem]=/usr/local/nagios/libexec/custom_check_mem -n $ARG1$
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_yum]=/usr/local/nagios/libexec/check_yum
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_apt]=/usr/local/nagios/libexec/check_apt
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_disk]=/usr/local/nagios/libexec/check_disk $ARG1$
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_ide_smart]=/usr/local/nagios/libexec/check_ide_smart $ARG1$
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_all_procs]=/usr/local/nagios/libexec/custom_check_procs
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_procs]=/usr/local/nagios/libexec/check_procs $ARG1$
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_open_files]=/usr/local/nagios/libexec/check_open_files.pl $ARG1$
Mar 14 19:51:54 tgcs018 nrpe[6604]: Added command[check_netstat]=/usr/local/nagios/libexec/check_netstat.pl -p $ARG1$ $ARG2$
Mar 14 19:51:54 tgcs018 nrpe[6604]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Mar 14 19:51:54 tgcs018 nrpe[6604]: Handling the connection...
Mar 14 19:51:54 tgcs018 nrpe[6604]: Host is asking for command 'check_mem' to be run...
Mar 14 19:51:54 tgcs018 nrpe[6604]: Running command: /usr/local/nagios/libexec/custom_check_mem -n -w 80% -c 90%
Mar 14 19:51:54 tgcs018 nrpe[6604]: Command completed with return code 0 and output: - 729 / 3791 MB (19%) Free Memory, Used: 3061 MB, Shared: 185 MB, Buffers + Cached: 384 MB | total=3791MB free=729MB used=3061MB shared=185MB buffers_and_cached=384MB
Mar 14 19:51:54 tgcs018 nrpe[6604]: Return Code: 0, Output: - 729 / 3791 MB (19%) Free Memory, Used: 3061 MB, Shared: 185 MB, Buffers + Cached: 384 MB | total=3791MB free=729MB used=3061MB shared=185MB buffers_and_cached=384MB
Mar 14 19:51:54 tgcs018 xinetd[20332]: EXIT: nrpe status=0 pid=6604 duration=0(sec)
Mar 14 19:52:01 tgcs018 systemd: Started Session 198866 of user nagios.
Mar 14 19:52:01 tgcs018 systemd: Starting Session 198866 of user nagios.
Mar 14 19:52:01 tgcs018 systemd: Started Session 198865 of user nagios.
Mar 14 19:52:01 tgcs018 systemd: Starting Session 198865 of user nagios.
Mar 14 19:52:39 tgcs018 xinetd[20332]: START: nrpe pid=6745 from=::ffff:10.2.8.79
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_asterisk]=/usr/local/nagios/libexec/check_asterisk.pl $ARG1$
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_sip]=/usr/local/nagios/libexec/check_sip $ARG1$
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_asterisk_sip_peers]=sudo /usr/local/nagios/libexec/check_asterisk_sip_peers.sh $ARG1$
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_asterisk_version]=/usr/local/nagios/libexec/nagisk.pl -c version
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_asterisk_peers]=/usr/local/nagios/libexec/nagisk.pl -c peers
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_asterisk_channels]=/usr/local/nagios/libexec/nagisk.pl -c channels
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_asterisk_zaptel]=/usr/local/nagios/libexec/nagisk.pl -c zaptel
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_asterisk_span]=/usr/local/nagios/libexec/nagisk.pl -c span -s 1
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_init_service]=sudo /usr/local/nagios/libexec/check_init_service $ARG1$
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_services]=/usr/local/nagios/libexec/check_services -p $ARG1$
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_users]=/usr/local/nagios/libexec/check_users $ARG1$
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_load]=/usr/local/nagios/libexec/check_load $ARG1$
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_swap]=/usr/local/nagios/libexec/check_swap $ARG1$
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_cpu_stats]=/usr/local/nagios/libexec/check_cpu_stats.sh $ARG1$
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_mem]=/usr/local/nagios/libexec/custom_check_mem -n $ARG1$
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_yum]=/usr/local/nagios/libexec/check_yum
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_apt]=/usr/local/nagios/libexec/check_apt
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_disk]=/usr/local/nagios/libexec/check_disk $ARG1$
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_ide_smart]=/usr/local/nagios/libexec/check_ide_smart $ARG1$
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_all_procs]=/usr/local/nagios/libexec/custom_check_procs
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_procs]=/usr/local/nagios/libexec/check_procs $ARG1$
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_open_files]=/usr/local/nagios/libexec/check_open_files.pl $ARG1$
Mar 14 19:52:39 tgcs018 nrpe[6745]: Added command[check_netstat]=/usr/local/nagios/libexec/check_netstat.pl -p $ARG1$ $ARG2$
Mar 14 19:52:39 tgcs018 nrpe[6745]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Mar 14 19:52:39 tgcs018 nrpe[6745]: Handling the connection...
Mar 14 19:52:39 tgcs018 nrpe[6745]: Host is asking for command '_NRPE_CHECK' to be run...
Mar 14 19:52:39 tgcs018 nrpe[6745]: Response: NRPE v2.15
Mar 14 19:52:39 tgcs018 nrpe[6745]: Return Code: 0, Output: NRPE v2.15
Mar 14 19:52:39 tgcs018 xinetd[20332]: EXIT: nrpe status=0 pid=6745 duration=0(sec)
Mar 14 19:52:53 tgcs018 xinetd[20332]: START: nrpe pid=6776 from=::ffff:10.2.8.79
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_asterisk]=/usr/local/nagios/libexec/check_asterisk.pl $ARG1$
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_sip]=/usr/local/nagios/libexec/check_sip $ARG1$
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_asterisk_sip_peers]=sudo /usr/local/nagios/libexec/check_asterisk_sip_peers.sh $ARG1$
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_asterisk_version]=/usr/local/nagios/libexec/nagisk.pl -c version
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_asterisk_peers]=/usr/local/nagios/libexec/nagisk.pl -c peers
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_asterisk_channels]=/usr/local/nagios/libexec/nagisk.pl -c channels
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_asterisk_zaptel]=/usr/local/nagios/libexec/nagisk.pl -c zaptel
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_asterisk_span]=/usr/local/nagios/libexec/nagisk.pl -c span -s 1
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_init_service]=sudo /usr/local/nagios/libexec/check_init_service $ARG1$
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_services]=/usr/local/nagios/libexec/check_services -p $ARG1$
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_users]=/usr/local/nagios/libexec/check_users $ARG1$
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_load]=/usr/local/nagios/libexec/check_load $ARG1$
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_swap]=/usr/local/nagios/libexec/check_swap $ARG1$
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_cpu_stats]=/usr/local/nagios/libexec/check_cpu_stats.sh $ARG1$
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_mem]=/usr/local/nagios/libexec/custom_check_mem -n $ARG1$
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_yum]=/usr/local/nagios/libexec/check_yum
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_apt]=/usr/local/nagios/libexec/check_apt
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_disk]=/usr/local/nagios/libexec/check_disk $ARG1$
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_ide_smart]=/usr/local/nagios/libexec/check_ide_smart $ARG1$
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_all_procs]=/usr/local/nagios/libexec/custom_check_procs
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_procs]=/usr/local/nagios/libexec/check_procs $ARG1$
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_open_files]=/usr/local/nagios/libexec/check_open_files.pl $ARG1$
Mar 14 19:52:53 tgcs018 nrpe[6776]: Added command[check_netstat]=/usr/local/nagios/libexec/check_netstat.pl -p $ARG1$ $ARG2$
Mar 14 19:52:53 tgcs018 nrpe[6776]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Mar 14 19:52:53 tgcs018 nrpe[6776]: Handling the connection...
Mar 14 19:52:53 tgcs018 nrpe[6776]: Host is asking for command 'check_mem' to be run...
Mar 14 19:52:53 tgcs018 nrpe[6776]: Running command: /usr/local/nagios/libexec/custom_check_mem -n -w 80% -c 90%
Mar 14 19:52:53 tgcs018 nrpe[6776]: Command completed with return code 0 and output: - 727 / 3791 MB (19%) Free Memory, Used: 3063 MB, Shared: 185 MB, Buffers + Cached: 385 MB | total=3791MB free=727MB used=3063MB shared=185MB buffers_and_cached=385MB
Mar 14 19:52:53 tgcs018 nrpe[6776]: Return Code: 0, Output: - 727 / 3791 MB (19%) Free Memory, Used: 3063 MB, Shared: 185 MB, Buffers + Cached: 385 MB | total=3791MB free=727MB used=3063MB shared=185MB buffers_and_cached=385MB
Mar 14 19:52:53 tgcs018 xinetd[20332]: EXIT: nrpe status=0 pid=6776 duration=0(sec)
Mar 14 19:53:01 tgcs018 systemd: Started Session 198868 of user nagios.
Mar 14 19:53:01 tgcs018 systemd: Starting Session 198868 of user nagios.
Mar 14 19:53:01 tgcs018 systemd: Started Session 198867 of user nagios.
Mar 14 19:53:01 tgcs018 systemd: Starting Session 198867 of user nagios.
Re: All Linux Server CPU Spike at same time
Posted: Wed Mar 15, 2017 4:31 pm
by ssax
What is the output of this command (run it from the XI server):
Code: Select all
/usr/local/nagios/libexec/check_nrpe -H 10.2.8.7 -c check_load -a '-w 5,10,15 -c 6,11,17'
Thank you
Re: All Linux Server CPU Spike at same time
Posted: Thu Mar 16, 2017 3:57 pm
by kwhogster
root@tgcs017:/usr/local/nagios/etc/objects# /usr/local/nagios/libexec/check_nrpe -H 10.2.8.7 -c check_load -a '-w 5,10,15 -c 6,11,17'
CHECK_NRPE: Socket timeout after 10 seconds.
root@tgcs017:/usr/local/nagios/etc/objects# /usr/local/nagios/libexec/check_nrpe -H 10.2.8.7 -t 90s -c check_load -a '-w 5,10,15 -c 6,11,17'
NRPE: Unable to read output
root@tgcs017:/usr/local/nagios/etc/objects# /usr/local/nagios/libexec/check_nrpe -H 10.2.8.7 -c check_load -a '-w 5,10,15 -c 6,11,17'
NRPE: Unable to read output
Re: All Linux Server CPU Spike at same time
Posted: Fri Mar 17, 2017 1:31 pm
by dwhitfield
You were using 10.2.8.74 and 10.2.8.79 but now are using 10.2.8.7. What are the IP addresses that are in play here? Is that just a typo? Did you actually make that typo when you ran the command?
Re: All Linux Server CPU Spike at same time
Posted: Fri Mar 17, 2017 1:38 pm
by kwhogster
My Linux hosts are this
10.2.8.74 Cent OS Nagios LogServer
10.2.8.79 Ubuntu Nagios Core 4.1
10.2.8.7 SUSE Enterprise vMA
The above are all VM's on different ESXi 6.0 Hosts
My other Linux is
10.2.8.72 RaspberryPi Test Nagios Core machine