Page 3 of 4
Re: Nagios XI Configuration Random Configuration Error
Posted: Thu Apr 09, 2015 6:48 am
by JakeHatMacys
jdalrymple wrote:JakeHatMacys wrote:Especially now that I did a restore to a clean config from 4/6 and it's still in the bad state.
What is a bad state? Is the monitoring engine still offline? When you say you restored from 4/6, did you do a full restore, or just reverted to a config snapshot? It sounds like full restore - which should negate any possible database issues.
Can you post the output of:
Code: Select all
/usr/local/nagiosxi/scripts/reconfigure_nagios.sh
Yeah the monitoring engine is still down, and I ran the restore script on a back up from 4/6:
/usr/local/nagiosxi/scripts/restore_xi.sh
And basically still no change.
Contenst of /usr/local/nagiosxi/scripts/reconfigure_nagios.sh:
Code: Select all
#!/bin/sh
# Copyright (c) 2008-2014 Nagios Enterprises, LLC. All rights reserved.
# $Id: reconfigure_nagios.sh 2854 2014-08-23 18:29:04Z swilkerson $
# exit codes:
# 1 config verification failed
# 2 nagiosql login failed
# 3 nagiosql import failed
# 4 reset_config_perms failed
# 5 nagiosql_exportall.php failed (write configs failed)
# 6 /etc/init.d/nagios restart failed
# 7 db_connect failed
#
# Import data to NagiosQL
./import_nagiosql.sh
ret=$?
if [ $ret -gt 0 ]; then
exit $ret
fi
# Restart Nagios
./restart_nagios_with_export.sh
ret=$?
if [ $ret -gt 0 ]; then
exit $ret
fi
exit 0
Also now when I run the Write Config gauntlet from top to bottom everything is fine but I get an error on the restart...
Capture.JPG
Re: Nagios XI Configuration Random Configuration Error
Posted: Thu Apr 09, 2015 10:15 am
by JakeHatMacys
Been digging around some of the logs to find more information to help.
Contents of /usr/local/nagiosxi/var/cmdsubsys.log
Code: Select all
more /usr/local/nagiosxi/var/cmdsubsys.log
PROCESSING COMMAND ID 383...
PROCESS COMMAND: CMD=11, DATA=a:1:{i:0;s:0:"";}
CMDLINE=/usr/local/nagiosxi/scripts/manage_services.sh start nagios
/usr/local/nagiosxi/scripts/manage_services.sh: fork: retry: Resource temporarily unavailable
Starting nagios:
Nagios Core 4.0.8
Copyright (c) 2009-present Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 08-12-2014
License: GPL
Website: http://www.nagios.org
Reading configuration data...
Read main config file okay...
Read object config files okay...
Running pre-flight check on configuration data...
Checking objects...
Error: There are no services defined!
Checked 0 services.
Error: There are no hosts defined!
Checked 0 hosts.
Checked 5 host groups.
Checked 0 service groups.
Checked 10 contacts.
Checked 2 contact groups.
Checked 122 commands.
Checked 16 time periods.
Checked 0 host escalations.
Checked 0 service escalations.
Checking for circular paths...
Checked 0 hosts
Checked 0 service dependencies
Checked 0 host dependencies
Checked 16 timeperiods
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...
Total Warnings: 0
Total Errors: 2
***> One or more problems was encountered while running the pre-flight check...
Check your configuration file(s) to ensure that they contain valid
directives and data defintions. If you are upgrading from a previous
version of Nagios, you should be aware that some variables/definitions
may have been removed or modified in this version. Make sure to read
the HTML documentation regarding the config files, as well as the
'Whats New' section to find out what has changed.
OUTPUT=
RETURNCODE=8
PROCESSING COMMAND ID 384...
PROCESS COMMAND: CMD=17, DATA=
APPLYING NAGIOSCORE CONFIG...
CMDLINE=cd /usr/local/nagiosxi/scripts && ./reconfigure_nagios.sh
URL: http://localhost/nagiosxi/includes/components/ccm/
CMDLINE
--2015-04-08 16:11:04-- http://localhost/nagiosxi/includes/components/ccm/
Resolving localhost... ::1, 127.0.0.1
Connecting to localhost|::1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: `nagiosql.login'
0K .......... 731K=0.01s
2015-04-08 16:11:04 (731 KB/s) - `nagiosql.login' saved [10831]
/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies http://localhost/nagiosxi/includes/components/ccm/
--no-check-certificate --post-data 'submit=Login&hidelog=true&loginSubmitted=true&username=nagiosxi&password=n@gweb' -O
nagiosql.loginLOGIN SUCCESSFUL!
IMPORTING CONFIG FILES...URL: http://localhost/nagiosxi/includes/components/ccm/
Array
(
)
RESETTING PERMS
SETUID ROOT OK
URL: http://localhost/nagiosxi/includes/components/ccm/
CMDLINE
--2015-04-08 16:11:05-- http://localhost/nagiosxi/includes/components/ccm/
Resolving localhost... ::1, 127.0.0.1
Connecting to localhost|::1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: `nagiosql.login'
0K .......... 744K=0.01s
2015-04-08 16:11:05 (744 KB/s) - `nagiosql.login' saved [10831]
/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies http://localhost/nagiosxi/includes/components/ccm/
--no-check-certificate --post-data 'submit=Login&hidelog=true&loginSubmitted=true&username=nagiosxi&password=n@gweb' -O
nagiosql.loginLOGIN SUCCESSFUL!
URL: http://localhost/nagiosxi/includes/components/ccm/
CMDLINE:
/usr/bin/wget --load-cookies=nagiosql.cookies http://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate
--post-data 'cmd=apply&type=writeConfig' -O nagiosql.export.monitoring
--2015-04-08 16:11:06-- http://localhost/nagiosxi/includes/components/ccm/
Resolving localhost... ::1, 127.0.0.1
Connecting to localhost|::1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: `nagiosql.export.monitoring'
0K .......... .......... .......... .......... .......... 151
50K .......... .......... .......... .......... .......... 134M
100K .......... .......... .......... .......... .......... 153M
150K .......... .......... .......... .......... .......... 164M
200K .......... .......... .......... .......... .......... 129M
250K .......... .......... .......... .......... .......... 171M
300K .......... ... 1.11M=5m39s
2015-04-08 16:16:44 (948 B/s) - `nagiosql.export.monitoring' saved [320984]
WRITE CONFIGS SUCCESSFUL!
./restart_nagios_with_export.sh: fork: retry: Resource temporarily unavailable
./restart_nagios_with_export.sh: fork: retry: Resource temporarily unavailable
./restart_nagios_with_export.sh: fork: retry: Resource temporarily unavailable
./restart_nagios_with_export.sh: fork: retry: Resource temporarily unavailable
./restart_nagios_with_export.sh: fork: Resource temporarily unavailable
OUTPUT=WRITE CONFIGS SUCCESSFUL!
RETURNCODE=254
Re: Nagios XI Configuration Random Configuration Error
Posted: Thu Apr 09, 2015 11:14 am
by cmerchant
Could you run the following to see the nagiosql Database count of hosts and services.
Code: Select all
echo 'select count(*) from nagios_hosts;select count(*) from nagios_services;' | mysql -u root -pnagiosxi nagios -t
also could show us the output of the following:
Re: Nagios XI Configuration Random Configuration Error
Posted: Thu Apr 09, 2015 11:33 am
by JakeHatMacys
cmerchant wrote:Could you run the following to see the nagiosql Database count of hosts and services.
Code: Select all
echo 'select count(*) from nagios_hosts;select count(*) from nagios_services;' | mysql -u root -pnagiosxi nagios -t
also could show us the output of the following:
Sure
Code: Select all
echo 'select count(*) from nagios_hosts;select count(*) from nagios_services;' | mysql -u root -pnagiosxi nagios -t
+----------+
| count(*) |
+----------+
| 2048 |
+----------+
+----------+
| count(*) |
+----------+
| 5380 |
+----------+
PS -efw (Everything I could capture after the command)
Code: Select all
nagios 28798 28795 0 Apr08 pts/368 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=310 [email protected] iosta
nagios 28800 28797 0 Apr08 pts/370 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=310 [email protected] iosta
nagios 28816 26260 0 Apr07 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 28818 28816 0 Apr07 pts/109 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] df -v
nagios 28819 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.7.238
nagios 28820 28819 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 28821 28820 0 Apr08 pts/191 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat
nagios 28830 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.120.38.2
nagios 28831 28830 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 28832 28831 0 Apr08 pts/280 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iosta
nagios 28944 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_linux_by_sshpass.sh 11.120.39.146
nagios 28946 28944 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 28947 28946 0 Apr08 pts/265 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=310 [email protected] iosta
nagios 28979 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.120.38.2
nagios 28980 28979 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 28981 28980 0 Apr08 pts/113 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iosta
nagios 29450 1 0 2014 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.27.87
nagios 29451 29450 0 2014 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 29452 29451 0 2014 pts/14 00:00:00 ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' ' ';
nagios 29525 1 0 Apr07 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_linux_by_sshpass.sh 11.48.27.45
nagios 29526 29525 0 Apr07 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 29527 29526 0 Apr07 pts/220 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=310 [email protected] iostat
nagios 29630 26956 0 Mar17 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 29632 29630 0 Mar17 pts/43 00:00:00 ssh -q -o StrictHostKeyChecking=no [email protected] df -v
nagios 29690 1 0 Apr07 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.45.6
nagios 29691 29690 0 Apr07 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 29692 29691 0 Apr07 pts/246 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -
apache 29716 21552 0 Apr08 ? 00:06:33 /usr/sbin/httpd
nagios 29981 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.221.7
nagios 29986 29981 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 29987 29986 0 Apr08 pts/273 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat
root 30014 1 0 Apr02 ? 00:00:00 sudo su - root
root 30015 2474 0 Apr02 ? 00:00:04 pmmasterd -ar
nagios 30227 1 0 Apr07 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.24.13
nagios 30228 30227 0 Apr07 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 30229 30228 0 Apr07 pts/124 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat
root 30298 2539 0 Jan15 ? 00:00:00 sshd: root@pts/29
root 30318 30298 0 Jan15 pts/29 00:00:00 -bash
nagios 30556 1 0 Mar17 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.168.2
nagios 30557 30556 0 Mar17 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 30558 30557 0 Mar17 pts/59 00:00:00 ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' '
nagios 30577 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.39.19
nagios 30578 30577 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 30579 30578 0 Apr08 pts/148 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat
nagios 30629 1 0 Mar20 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_linux_by_sshpass.sh 11.48.115.33
nagios 30636 30629 0 Mar20 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 30647 30636 0 Mar20 pts/71 00:00:00 ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' ' '
nagios 30653 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_linux_by_sshpass.sh 11.48.221.70
nagios 30654 30653 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 30655 30654 0 Apr08 pts/293 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=310 [email protected] iostat
nagios 30662 1 0 Apr07 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_linux_by_sshpass.sh 11.120.39.146
nagios 30672 30662 0 Apr07 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 30676 30672 0 Apr07 pts/217 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=310 [email protected] iosta
nagios 30738 1 0 Apr07 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.45.11
nagios 30739 30738 0 Apr07 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 30740 30739 0 Apr07 pts/161 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat
nagios 30761 1 0 2014 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.27.87
nagios 30766 30761 0 2014 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 30768 30766 0 2014 pts/52 00:00:00 ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' ' ';
nagios 30874 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.221.1
nagios 30876 30874 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 30877 30876 0 Apr08 pts/214 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iosta
nagios 30967 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.120.114.
nagios 30968 30967 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 30969 30968 0 Apr08 pts/153 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iosta
nagios 30970 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.115.3
nagios 30971 30970 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 30972 30971 0 Apr08 pts/166 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat
postgres 30983 21530 0 Apr08 ? 00:00:07 postgres: nagiosxi nagiosxi [local] idle
nagios 31037 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.27.42
nagios 31040 31037 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 31041 31040 0 Apr08 pts/178 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat
root 31195 30014 0 Apr02 ? 00:00:00 sudo su - root
root 31196 31195 0 Apr02 ? 00:00:00 su - root
root 31198 31196 0 Apr02 ? 00:00:00 -bash
root 31210 28576 0 Apr02 ? 00:00:00 sudo su - root
root 31213 31210 0 Apr02 ? 00:00:00 su - root
root 31241 31213 0 Apr02 ? 00:00:00 -bash
nagios 31248 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.27.41
nagios 31249 31248 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 31250 31249 0 Apr08 pts/257 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat
nagios 31257 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.115.3
nagios 31258 31257 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 31259 31258 0 Apr08 pts/122 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat
root 31334 1 0 Apr02 ? 00:00:00 sudo su - root
root 31335 2474 0 Apr02 ? 00:00:04 pmmasterd -ar
nagios 31341 1 0 Apr07 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.21.11
nagios 31342 31341 0 Apr07 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 31343 31342 0 Apr07 pts/204 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat
nagios 31485 1 0 Jan07 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_linux_by_sshpass.sh 11.120.114.58
nagios 31500 31485 0 Jan07 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 31511 31500 0 Jan07 pts/60 00:00:00 ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' '
nagios 31980 1 0 Jan29 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_linux_by_sshpass.sh 11.48.45.116
nagios 31997 31980 0 Jan29 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 32007 31997 0 Jan29 pts/91 00:00:00 ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' ' '
nagios 32035 1 0 Mar17 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_linux_by_sshpass.sh 11.122.80.14
nagios 32037 32035 0 Mar17 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 32038 32037 0 Mar17 pts/93 00:00:00 ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' ' '
nagios 32058 1 0 Mar17 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_linux_by_sshpass.sh 11.120.114.58
nagios 32059 32058 0 Mar17 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 32060 32059 0 Mar17 pts/111 00:00:00 ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' '
nagios 32097 1 0 2014 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.120.114.
nagios 32104 32097 0 2014 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 32107 32104 0 2014 pts/19 00:00:00 ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' '
nagios 32130 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_linux_by_sshpass.sh 11.120.113.102
nagios 32131 32130 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 32132 32131 0 Apr08 pts/425 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=310 [email protected] iost
nagios 32133 1 0 Apr07 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.45.6
nagios 32134 32133 0 Apr07 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 32135 32134 0 Apr07 pts/205 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -
nagios 32550 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.21.12
nagios 32551 32550 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 32552 32551 0 Apr08 pts/382 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat
root 32615 1 0 Mar06 ? 00:00:00 /bin/sh /opt/OV/HPBsmIntNagios/genint/bin/genintd.sh /opt/OV/HPBsmIntNagios/genint/c
nagios 32617 1 0 Apr07 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.115.3
root 32619 32615 0 Mar06 ? 04:47:21 /opt/OV/HPBsmIntNagios/genint/bin/genintd /opt/OV/HPBsmIntNagios/genint/conf/nagios_
nagios 32622 32617 0 Apr07 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no
nagios 32624 32622 0 Apr07 pts/145 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat
df -h
Code: Select all
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/localvg00-lv_slash
17G 6.3G 9.2G 41% /
tmpfs 48G 1.0M 48G 1% /dev/shm
/dev/mapper/localvg00-lv_archive
4.0G 194M 3.6G 6% /archive
/dev/sda1 248M 38M 198M 17% /boot
/dev/mapper/localvg00-lv_home
4.0G 1.4G 2.4G 37% /home
/dev/mapper/localvg00-lv_opt
16G 427M 15G 3% /opt
/dev/mapper/localvg00-lv_tmp
3.0G 652M 2.2G 23% /tmp
/dev/mapper/localvg00-lv_usr
9.9G 2.4G 7.1G 25% /usr
/dev/mapper/localvg00-lv_usr_local
56G 13G 41G 24% /usr/local
/dev/mapper/localvg00-lv_var
9.9G 4.7G 4.8G 50% /var
df -I:
Code: Select all
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/mapper/localvg00-lv_slash
1081344 7362 1073982 1% /
tmpfs 12381499 2 12381497 1% /dev/shm
/dev/mapper/localvg00-lv_archive
262144 225 261919 1% /archive
/dev/sda1 65536 39 65497 1% /boot
/dev/mapper/localvg00-lv_home
262144 1090 261054 1% /home
/dev/mapper/localvg00-lv_opt
1048576 2907 1045669 1% /opt
/dev/mapper/localvg00-lv_tmp
196608 7808 188800 4% /tmp
/dev/mapper/localvg00-lv_usr
655360 78731 576629 13% /usr
/dev/mapper/localvg00-lv_usr_local
3670016 35415 3634601 1% /usr/local
/dev/mapper/localvg00-lv_var
655360 5317 650043 1% /var
Re: Nagios XI Configuration Random Configuration Error
Posted: Thu Apr 09, 2015 12:10 pm
by cmerchant
What is the status of the The Monitoring engine? Is everything "green"?
Can you show us the permissions on the following:
If you can repeat the process of Write Config, Verify Config (ONLY), then go to the command line and do the following:
Code: Select all
cd /usr/local/nagiosxi/scripts
./reconfigure_nagios.sh
Re: Nagios XI Configuration Random Configuration Error
Posted: Thu Apr 09, 2015 1:32 pm
by JakeHatMacys
cmerchant wrote:What is the status of the The Monitoring engine? Is everything "green"?
Can you show us the permissions on the following:
If you can repeat the process of Write Config, Verify Config (ONLY), then go to the command line and do the following:
Code: Select all
cd /usr/local/nagiosxi/scripts
./reconfigure_nagios.sh
Everything is red now...
Capture.JPG
ls -lR /usr/local/nagios/etc/
Code: Select all
ls -lR /usr/local/nagios/etc/
/usr/local/nagios/etc/:
total 376
-rw-rw-r-- 1 apache nagios 2132 Apr 1 15:38 cgi.cfg
-rw-r--r-- 1 apache nagios 25651 Apr 9 09:14 commands.cfg
-rw-r--r-- 1 apache nagios 1106 Apr 9 09:14 contactgroups.cfg
-rw-r--r-- 1 apache nagios 4758 Apr 9 09:14 contacts.cfg
-rw-r--r-- 1 apache nagios 1675 Apr 9 09:14 contacttemplates.cfg
drwxrwsr-x 7 apache nagios 4096 Jan 29 16:57 etc_back2
-rw-r--r-- 1 apache nagios 817 Apr 9 09:14 hostdependencies.cfg
-rw-r--r-- 1 apache nagios 819 Apr 9 09:14 hostescalations.cfg
-rw-r--r-- 1 apache nagios 837 Apr 9 09:14 hostextinfo.cfg
-rw-r--r-- 1 apache nagios 1475 Apr 9 09:14 hostgroups.cfg
drwsrwsr-x 2 apache nagios 86016 Apr 9 09:14 hosts
-rw-r--r-- 1 apache nagios 15516 Apr 9 09:14 hosttemplates.cfg
drwsrwsr-x 2 apache nagios 4096 Apr 1 15:38 import
-rwxrwxr-x 1 apache nagios 5629 Mar 5 17:24 nagios.cfg
-rwxrwxr-x 1 apache nagios 5597 Dec 4 19:36 nagios.cfg.xi
-rw-rw-r-- 1 apache nagios 2229 Dec 4 19:17 ndo2db.cfg
-rw-rw-r-- 1 apache nagios 4827 Dec 4 19:17 ndomod.cfg
-rw-rw-r-- 1 apache nagios 7989 Mar 25 14:00 nrpe.cfg
-rw-rw-r-- 1 apache nagios 5345 Dec 4 19:17 nsca.cfg
drwxrwsr-x 4 apache nagios 4096 Jan 29 16:57 pnp
-rw-rw-r-- 1 apache nagios 0 Mar 17 22:23 recurringdowntime.cfg
-rwxrwxr-x 1 apache nagios 210 Dec 4 19:15 resource.cfg
-rwxrwxr-x 1 apache nagios 210 Dec 4 19:36 resource.cfg.xi
-rw-rw-r-- 1 apache nagios 1627 Dec 4 19:17 send_nsca.cfg
-rw-r--r-- 1 apache nagios 823 Apr 9 09:14 servicedependencies.cfg
-rw-r--r-- 1 apache nagios 825 Apr 9 09:14 serviceescalations.cfg
-rw-r--r-- 1 apache nagios 843 Apr 9 09:14 serviceextinfo.cfg
-rw-r--r-- 1 apache nagios 813 Apr 9 09:14 servicegroups.cfg
drwsrwsr-x 2 apache nagios 86016 Apr 9 09:14 services
-rw-r--r-- 1 apache nagios 22867 Apr 9 09:14 servicetemplates.cfg
drwsrwsr-x 2 apache nagios 4096 Apr 8 15:49 static
-rw-r--r-- 1 apache nagios 7247 Apr 9 09:14 timeperiods.cfg
/usr/local/nagios/etc/etc_back2:
total 32
drwxrwsr-x 2 apache nagios 4096 Dec 4 19:46 hosts
drwxrwsr-x 2 apache nagios 4096 Dec 4 19:31 import
-rwxrwxr-x 1 apache nagios 5597 Dec 4 19:31 nagios.cfg.xi
drwxrwsr-x 4 apache nagios 4096 Jan 29 16:57 pnp
-rwxrwxr-x 1 apache nagios 210 Dec 4 19:31 resource.cfg.xi
drwxrwsr-x 2 apache nagios 4096 Dec 4 19:46 services
drwxrwsr-x 2 apache nagios 4096 Dec 4 19:46 static
/usr/local/nagios/etc/etc_back2/hosts:
total 0
/usr/local/nagios/etc/etc_back2/import:
total 0
/usr/local/nagios/etc/etc_back2/pnp:
total 40
-rw-rw-r-- 1 apache nagios 6952 Dec 4 19:31 background.pdf
drwxrwsr-x 2 apache nagios 4096 Jan 29 16:57 check_commands
-rw-rw-r-- 1 apache nagios 3417 Dec 4 19:31 config.php
-rw-rw-r-- 1 apache nagios 4127 Dec 4 19:31 npcd.cfg-sample
drwxrwsr-x 2 apache nagios 4096 Jan 29 16:57 pages
-rw-rw-r-- 1 apache nagios 63 Dec 4 19:31 pnp4nagios_release
-rw-rw-r-- 1 apache nagios 793 Dec 4 19:31 process_perfdata.cfg-sample
-rw-rw-r-- 1 apache nagios 524 Dec 4 19:31 rra.cfg-sample
/usr/local/nagios/etc/etc_back2/pnp/check_commands:
total 4
-rw-rw-r-- 1 apache nagios 755 Dec 4 19:31 check_nwstat.cfg-sample
/usr/local/nagios/etc/etc_back2/pnp/pages:
total 4
-rw-rw-r-- 1 apache nagios 324 Dec 4 19:31 web_traffic.cfg-sample
/usr/local/nagios/etc/etc_back2/services:
total 0
/usr/local/nagios/etc/etc_back2/static:
total 0
/usr/local/nagios/etc/hosts:
total 0
/usr/local/nagios/etc/import:
total 0
/usr/local/nagios/etc/pnp:
total 52
-rw-rw-r-- 1 apache nagios 6952 Jan 19 13:24 background.pdf
drwxrwsr-x 2 apache nagios 4096 Jan 29 16:57 check_commands
-rw-rw-r-- 1 apache nagios 4076 Jan 19 13:24 config.php
-rw-rw-r-- 1 apache nagios 3090 Jan 19 13:24 npcd.cfg
-rw-rw-r-- 1 apache nagios 4127 Dec 4 19:17 npcd.cfg-sample
drwxrwsr-x 2 apache nagios 4096 Jan 29 16:57 pages
-rw-rw-r-- 1 apache nagios 63 Jan 19 13:24 pnp4nagios_release
-rw-rw-r-- 1 apache nagios 793 Dec 4 19:17 process_perfdata.cfg
-rw-rw-r-- 1 apache nagios 793 Dec 4 19:17 process_perfdata.cfg-sample
-rw-rw-r-- 1 apache nagios 524 Dec 4 19:17 rra.cfg
-rw-rw-r-- 1 apache nagios 524 Dec 4 19:17 rra.cfg-sample
/usr/local/nagios/etc/pnp/check_commands:
total 4
-rw-rw-r-- 1 apache nagios 755 Dec 4 19:17 check_nwstat.cfg-sample
/usr/local/nagios/etc/pnp/pages:
total 4
-rw-rw-r-- 1 apache nagios 324 Dec 4 19:17 web_traffic.cfg-sample
/usr/local/nagios/etc/services:
total 0
/usr/local/nagios/etc/static:
total 8
-rwxrwxr-x 1 apache nagios 878 Feb 6 15:21 xiobjects.cfg
-rwxrwxr-x 1 apache nagios 4002 Feb 6 15:21 xitemplates.cfg
-rwxrwxr-x 1 apache nagios 0 Feb 6 15:21 xitest.cfg
[root@esu1l268 ~]# grep nag /etc/group
nagios

59996:nagios,apache
nagcmd

59997:nagios,apache
ps -efw|grep [n]agios
Code: Select all
nagios 29527 29526 0 Apr07 pts/220 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=310 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 29630 26956 0 Mar17 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no [email protected] df -v
nagios 29632 29630 0 Mar17 pts/43 00:00:00 ssh -q -o StrictHostKeyChecking=no [email protected] df -v
nagios 29690 1 0 Apr07 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.45.6
nagios 29691 29690 0 Apr07 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 29692 29691 0 Apr07 pts/246 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 29981 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.221.75
nagios 29986 29981 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 29987 29986 0 Apr08 pts/273 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30227 1 0 Apr07 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.24.134
nagios 30228 30227 0 Apr07 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30229 30228 0 Apr07 pts/124 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30556 1 0 Mar17 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.168.224
nagios 30557 30556 0 Mar17 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30558 30557 0 Mar17 pts/59 00:00:00 ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30577 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.39.198
nagios 30578 30577 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30579 30578 0 Apr08 pts/148 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30629 1 0 Mar20 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_linux_by_sshpass.sh 11.48.115.33
nagios 30636 30629 0 Mar20 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30647 30636 0 Mar20 pts/71 00:00:00 ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30653 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_linux_by_sshpass.sh 11.48.221.70
nagios 30654 30653 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=310 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30655 30654 0 Apr08 pts/293 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=310 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30662 1 0 Apr07 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_linux_by_sshpass.sh 11.120.39.146
nagios 30672 30662 0 Apr07 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=310 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30676 30672 0 Apr07 pts/217 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=310 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30738 1 0 Apr07 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.45.115
nagios 30739 30738 0 Apr07 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30740 30739 0 Apr07 pts/161 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30761 1 0 2014 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.27.87
nagios 30766 30761 0 2014 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30768 30766 0 2014 pts/52 00:00:00 ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30783 2930 0 14:02 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.120.38.228
nagios 30787 30783 0 14:02 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30789 30787 0 14:02 pts/335 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30874 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.221.116
nagios 30876 30874 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30877 30876 0 Apr08 pts/214 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30967 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.120.114.58
nagios 30968 30967 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30969 30968 0 Apr08 pts/153 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30970 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.115.32
nagios 30971 30970 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 30972 30971 0 Apr08 pts/166 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
postgres 30983 21530 0 Apr08 ? 00:00:08 postgres: nagiosxi nagiosxi [local] idle
nagios 31037 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.27.42
nagios 31040 31037 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 31041 31040 0 Apr08 pts/178 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 31248 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.27.41
nagios 31249 31248 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 31250 31249 0 Apr08 pts/257 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 31257 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.115.32
nagios 31258 31257 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 31259 31258 0 Apr08 pts/122 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 31341 1 0 Apr07 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.21.118
nagios 31342 31341 0 Apr07 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 31343 31342 0 Apr07 pts/204 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 31485 1 0 Jan07 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_linux_by_sshpass.sh 11.120.114.58
nagios 31500 31485 0 Jan07 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 31511 31500 0 Jan07 pts/60 00:00:00 ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 31696 2961 0 14:02 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_AIX_by_sshpass.sh 11.48.25.162
nagios 31698 31696 0 14:02 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -t 5 2 | sed -e 's/,/./g'|tr -s ' ' ';' | tail -1
nagios 31702 31698 0 14:02 pts/392 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -t 5 2 | sed -e 's/,/./g'|tr -s ' ' ';' | tail -1
nagios 31784 2929 0 14:02 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_AIX_by_sshpass.sh 11.122.16.42
nagios 31786 31784 0 14:02 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -t 5 2 | sed -e 's/,/./g'|tr -s ' ' ';' | tail -1
nagios 31787 31786 0 14:02 pts/358 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -t 5 2 | sed -e 's/,/./g'|tr -s ' ' ';' | tail -1
nagios 31788 2931 0 14:02 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_AIX_by_sshpass.sh 11.120.113.52
nagios 31789 31788 0 14:02 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -t 5 2 | sed -e 's/,/./g'|tr -s ' ' ';' | tail -1
nagios 31790 31789 0 14:02 pts/420 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -t 5 2 | sed -e 's/,/./g'|tr -s ' ' ';' | tail -1
nagios 31953 2941 0 14:02 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_AIX_by_sshpass.sh 11.48.114.50
nagios 31954 31953 0 14:02 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -t 5 2 | sed -e 's/,/./g'|tr -s ' ' ';' | tail -1
nagios 31955 31954 0 14:02 pts/463 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -t 5 2 | sed -e 's/,/./g'|tr -s ' ' ';' | tail -1
nagios 31980 1 0 Jan29 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_linux_by_sshpass.sh 11.48.45.116
nagios 31997 31980 0 Jan29 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 32007 31997 0 Jan29 pts/91 00:00:00 ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 32035 1 0 Mar17 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_linux_by_sshpass.sh 11.122.80.14
nagios 32037 32035 0 Mar17 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 32038 32037 0 Mar17 pts/93 00:00:00 ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 32058 1 0 Mar17 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_linux_by_sshpass.sh 11.120.114.58
nagios 32059 32058 0 Mar17 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 32060 32059 0 Mar17 pts/111 00:00:00 ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 32097 1 0 2014 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.120.114.58
nagios 32104 32097 0 2014 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 32107 32104 0 2014 pts/19 00:00:00 ssh -q -o StrictHostKeyChecking=no [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 32130 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_FS_space_linux_by_sshpass.sh 11.120.113.102
nagios 32131 32130 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=310 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 32132 32131 0 Apr08 pts/425 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=310 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 32133 1 0 Apr07 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.45.6
nagios 32134 32133 0 Apr07 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 32135 32134 0 Apr07 pts/205 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 32550 1 0 Apr08 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.21.126
nagios 32551 32550 0 Apr08 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 32552 32551 0 Apr08 pts/382 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
root 32615 1 0 Mar06 ? 00:00:00 /bin/sh /opt/OV/HPBsmIntNagios/genint/bin/genintd.sh /opt/OV/HPBsmIntNagios/genint/conf/nagios_server.conf
nagios 32617 1 0 Apr07 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.48.115.37
root 32619 32615 0 Mar06 ? 04:47:54 /opt/OV/HPBsmIntNagios/genint/bin/genintd /opt/OV/HPBsmIntNagios/genint/conf/nagios_server.conf
nagios 32622 32617 0 Apr07 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 32624 32622 0 Apr07 pts/145 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 32724 2960 0 14:02 ? 00:00:00 /bin/sh /usr/local/nagios/libexec/check_read_only_fs_linux_by_sshpass.sh 11.120.113.16
nagios 32725 32724 0 14:02 ? 00:00:00 /usr/local/nagios/sshpass-1.05/sshpass -pzzzzzzz ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
nagios 32726 32725 0 14:02 pts/415 00:00:00 ssh -q -o StrictHostKeyChecking=no -o ConnectTimeout=300 [email protected] iostat -c 5 2 | tr -s ' ' ';' | sed '/^$/d' | tail -1
[root@esu1l268 ~]#
Reconfigure is still running but it looks like it's stuck on this:
./reconfigure_nagios.sh
Another reconfigure process is still running, sleeping...
NM it finally went through. Shows all the warning for no contacts and then this:
Code: Select all
Checked 2048 hosts.
Checked 5 host groups.
Checked 0 service groups.
Checked 10 contacts.
Checked 2 contact groups.
Checked 122 commands.
Checked 16 time periods.
Checked 0 host escalations.
Checked 0 service escalations.
Checking for circular paths...
Checked 2048 hosts
Checked 0 service dependencies
Checked 0 host dependencies
Checked 16 timeperiods
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...
Total Warnings: 7419
Total Errors: 0
Things look okay - No serious problems were detected during the pre-flight check
RET: 0
Running configuration check...
Stopping nagios:. done.
Starting nagios: done.
And after running this still no changes :/ still red
Re: Nagios XI Configuration Random Configuration Error
Posted: Thu Apr 09, 2015 3:48 pm
by cmerchant
Noticed the redness on your monitoring engine status... I think in order to have the apply configuration work, we need to have nagios, ndo2db restarted.
But I would check the health of your database services as well.
Code: Select all
service nagios stop
service nagios status
service ndo2db stop
service ndo2db status
service mysqld stop
service mysqld status
service mysqld start
tail -100 /var/log/mysqld.log
service ndo2db start
tail -100 /var/log/messages
service nagios start
tail -100 /usr/local/nagios/var/nagios.log
after all of that, could you verify the status of the monitoring engine status.
Code: Select all
cat /usr/local/nagiosxi/var/sysstat.log
Re: Nagios XI Configuration Random Configuration Error
Posted: Thu Apr 09, 2015 6:00 pm
by JakeHatMacys
cmerchant wrote:Noticed the redness on your monitoring engine status... I think in order to have the apply configuration work, we need to have nagios, ndo2db restarted.
But I would check the health of your database services as well.
Code: Select all
service nagios stop
service nagios status
service ndo2db stop
service ndo2db status
service mysqld stop
service mysqld status
service mysqld start
tail -100 /var/log/mysqld.log
service ndo2db start
tail -100 /var/log/messages
service nagios start
tail -100 /usr/local/nagios/var/nagios.log
after all of that, could you verify the status of the monitoring engine status.
Code: Select all
cat /usr/local/nagiosxi/var/sysstat.log
Did all that, and still Red. Applying config just clocks endlessly now.
This is the result of cat /usr/local/nagiosxi/var/sysstat.log:
Code: Select all
[min_execution_time] => 0.00127
[max_execution_time] => 60.017
[avg_execution_time] => 8.1480894702602
)
)
PHP Warning: exec(): Unable to fork [/usr/bin/uptime | sed s/,//g | awk -F'average: ' '{ print $2 }'] in /usr/local/nagiosxi/cron/sysstat.php on line 198
PHP Notice: Undefined offset: 0 in /usr/local/nagiosxi/cron/sysstat.php on line 201
PHP Notice: Undefined offset: 1 in /usr/local/nagiosxi/cron/sysstat.php on line 205
PHP Notice: Undefined offset: 2 in /usr/local/nagiosxi/cron/sysstat.php on line 206
LOAD:
Array
(
[load1] =>
[load5] =>
[load15] =>
)
PHP Warning: exec(): Unable to fork [/usr/bin/free -m | head --lines=2 | tail --lines=1 | awk '{ print $2,$3,$4,$5,$6,$7}'] in /usr/local/nagiosxi/cron/sysstat.php on line 220
PHP Notice: Undefined offset: 0 in /usr/local/nagiosxi/cron/sysstat.php on line 223
PHP Notice: Undefined offset: 1 in /usr/local/nagiosxi/cron/sysstat.php on line 227
PHP Notice: Undefined offset: 2 in /usr/local/nagiosxi/cron/sysstat.php on line 228
PHP Notice: Undefined offset: 3 in /usr/local/nagiosxi/cron/sysstat.php on line 229
PHP Notice: Undefined offset: 4 in /usr/local/nagiosxi/cron/sysstat.php on line 230
PHP Notice: Undefined offset: 5 in /usr/local/nagiosxi/cron/sysstat.php on line 231
MEMORY:
Array
(
[total] =>
[used] =>
[free] =>
[shared] =>
[buffers] =>
[cached] =>
)
PHP Warning: exec(): Unable to fork [/usr/bin/free -m | tail --lines=1 | awk '{ print $2,$3,$4}'] in /usr/local/nagiosxi/cron/sysstat.php on line 244
PHP Notice: Undefined offset: 0 in /usr/local/nagiosxi/cron/sysstat.php on line 247
PHP Notice: Undefined offset: 1 in /usr/local/nagiosxi/cron/sysstat.php on line 251
PHP Notice: Undefined offset: 2 in /usr/local/nagiosxi/cron/sysstat.php on line 252
SWAP:
Array
(
[total] =>
[used] =>
[free] =>
)
PHP Warning: exec(): Unable to fork [/usr/bin/iostat -c 5 2 | tail --lines=2 | head --lines=1 | awk '{ print $1,$2,$3,$4,$5,$6 }'] in /usr/local/nagiosxi/cron/sysstat.php on line 265
PHP Notice: Undefined offset: 0 in /usr/local/nagiosxi/cron/sysstat.php on line 268
PHP Notice: Undefined offset: 1 in /usr/local/nagiosxi/cron/sysstat.php on line 272
PHP Notice: Undefined offset: 2 in /usr/local/nagiosxi/cron/sysstat.php on line 273
PHP Notice: Undefined offset: 3 in /usr/local/nagiosxi/cron/sysstat.php on line 274
PHP Notice: Undefined offset: 4 in /usr/local/nagiosxi/cron/sysstat.php on line 275
PHP Notice: Undefined offset: 5 in /usr/local/nagiosxi/cron/sysstat.php on line 276
IOSTAT:
Array
(
[user] =>
[nice] =>
[system] =>
[iowait] =>
[steal] =>
[idle] =>
)
.DB BACKEND:
Array
(
[last_checkin] => 2015-04-08 14:40:59
[bytes_processed] => 18156270
[entries_processed] => 38119
[connect_time] => 2015-04-08 14:36:55
[disconnect_time] => 0000-00-00 00:00:00
)
CMDLINE=/etc/init.d/nagios status
PHP Warning: system(): Unable to fork [/etc/init.d/nagios status] in /usr/local/nagiosxi/cron/sysstat.php on line 159
OUTPUT=
RETURNCODE=-1
CMDLINE=/etc/init.d/npcd status
PHP Warning: system(): Unable to fork [/etc/init.d/npcd status] in /usr/local/nagiosxi/cron/sysstat.php on line 159
OUTPUT=
RETURNCODE=-1
CMDLINE=/etc/init.d/ndo2db status
PHP Warning: system(): Unable to fork [/etc/init.d/ndo2db status] in /usr/local/nagiosxi/cron/sysstat.php on line 159
OUTPUT=
RETURNCODE=-1
DAEMONS:
Array
(
[nagioscore] => Array
(
[daemon] => nagios
[output] =>
[return_code] => -1
[status] => 1
)
[pnp] => Array
(
[daemon] => npcd
[output] =>
[return_code] => -1
[status] => 1
)
[ndoutils] => Array
(
[daemon] => ndo2db
[output] =>
[return_code] => -1
[status] => 1
)
)
CORE STATS:
Array
(
[activehostchecks] => Array
(
[1min] => 235
[5min] => 1144
[15min] => 1144
)
[passivehostchecks] => Array
(
[1min] => 0
[5min] => 0
[15min] => 0
)
[activeservicechecks] => Array
(
[1min] => 341
[5min] => 1440
[15min] => 1440
)
[passiveservicechecks] => Array
(
[1min] => 0
[5min] => 0
[15min] => 0
)
[activehostcheckperf] => Array
(
[min_latency] => 0
[max_latency] => 1.72661
[avg_latency] => 0.03091142578125
[min_execution_time] => 0.00167
[max_execution_time] => 10.00291
[avg_execution_time] => 2.8069380517578
)
[activeservicecheckperf] => Array
(
[min_latency] => 0
[max_latency] => 7.754
[avg_latency] => 0.0664435724907063
[min_execution_time] => 0.00127
[max_execution_time] => 60.017
[avg_execution_time] => 8.1498299758364
)
)
PHP Warning: exec(): Unable to fork [/usr/bin/uptime | sed s/,//g | awk -F'average: ' '{ print $2 }'] in /usr/local/nagiosxi/cron/sysstat.php on line 198
PHP Notice: Undefined offset: 0 in /usr/local/nagiosxi/cron/sysstat.php on line 201
PHP Notice: Undefined offset: 1 in /usr/local/nagiosxi/cron/sysstat.php on line 205
PHP Notice: Undefined offset: 2 in /usr/local/nagiosxi/cron/sysstat.php on line 206
LOAD:
Array
(
[load1] =>
[load5] =>
[load15] =>
)
PHP Warning: exec(): Unable to fork [/usr/bin/free -m | head --lines=2 | tail --lines=1 | awk '{ print $2,$3,$4,$5,$6,$7}'] in /usr/local/nagiosxi/cron/sysstat.php on line 220
PHP Notice: Undefined offset: 0 in /usr/local/nagiosxi/cron/sysstat.php on line 223
PHP Notice: Undefined offset: 1 in /usr/local/nagiosxi/cron/sysstat.php on line 227
PHP Notice: Undefined offset: 2 in /usr/local/nagiosxi/cron/sysstat.php on line 228
PHP Notice: Undefined offset: 3 in /usr/local/nagiosxi/cron/sysstat.php on line 229
PHP Notice: Undefined offset: 4 in /usr/local/nagiosxi/cron/sysstat.php on line 230
PHP Notice: Undefined offset: 5 in /usr/local/nagiosxi/cron/sysstat.php on line 231
MEMORY:
Array
(
[total] =>
[used] =>
[free] =>
[shared] =>
[buffers] =>
[cached] =>
)
PHP Warning: exec(): Unable to fork [/usr/bin/free -m | tail --lines=1 | awk '{ print $2,$3,$4}'] in /usr/local/nagiosxi/cron/sysstat.php on line 244
PHP Notice: Undefined offset: 0 in /usr/local/nagiosxi/cron/sysstat.php on line 247
PHP Notice: Undefined offset: 1 in /usr/local/nagiosxi/cron/sysstat.php on line 251
PHP Notice: Undefined offset: 2 in /usr/local/nagiosxi/cron/sysstat.php on line 252
SWAP:
Array
(
[total] =>
[used] =>
[free] =>
)
PHP Warning: exec(): Unable to fork [/usr/bin/iostat -c 5 2 | tail --lines=2 | head --lines=1 | awk '{ print $1,$2,$3,$4,$5,$6 }'] in /usr/local/nagiosxi/cron/sysstat.php on line 265
PHP Notice: Undefined offset: 0 in /usr/local/nagiosxi/cron/sysstat.php on line 268
PHP Notice: Undefined offset: 1 in /usr/local/nagiosxi/cron/sysstat.php on line 272
PHP Notice: Undefined offset: 2 in /usr/local/nagiosxi/cron/sysstat.php on line 273
PHP Notice: Undefined offset: 3 in /usr/local/nagiosxi/cron/sysstat.php on line 274
PHP Notice: Undefined offset: 4 in /usr/local/nagiosxi/cron/sysstat.php on line 275
PHP Notice: Undefined offset: 5 in /usr/local/nagiosxi/cron/sysstat.php on line 276
IOSTAT:
Array
(
[user] =>
[nice] =>
[system] =>
[iowait] =>
[steal] =>
[idle] =>
)
.Done
[root@esu1l268 ~]#
tailing the ndo2db:
Code: Select all
141211 9:10:24 InnoDB: Completed initialization of buffer pool
141211 9:10:24 InnoDB: Started; log sequence number 0 44263
141211 9:10:24 [Note] Event Scheduler: Loaded 0 events
141211 9:10:24 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.67' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution
141212 15:33:10 [Note] /usr/libexec/mysqld: Normal shutdown
141212 15:33:10 [Note] Event Scheduler: Purging the queue. 0 events
141212 15:33:12 InnoDB: Starting shutdown...
141212 15:33:16 InnoDB: Shutdown completed; log sequence number 0 44273
141212 15:33:16 [Note] /usr/libexec/mysqld: Shutdown complete
141212 15:33:16 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
141212 15:33:17 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
141212 15:33:17 InnoDB: Initializing buffer pool, size = 8.0M
141212 15:33:17 InnoDB: Completed initialization of buffer pool
141212 15:33:17 InnoDB: Started; log sequence number 0 44273
141212 15:33:17 [Note] Event Scheduler: Loaded 0 events
141212 15:33:17 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.67' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution
141212 15:48:29 [Note] /usr/libexec/mysqld: Normal shutdown
141212 15:48:29 [Note] Event Scheduler: Purging the queue. 0 events
141212 15:48:31 InnoDB: Starting shutdown...
141212 15:48:33 InnoDB: Shutdown completed; log sequence number 0 44273
141212 15:48:33 [Note] /usr/libexec/mysqld: Shutdown complete
141212 15:48:33 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
141212 15:54:33 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
141212 15:54:33 InnoDB: Initializing buffer pool, size = 8.0M
141212 15:54:33 InnoDB: Completed initialization of buffer pool
141212 15:54:33 InnoDB: Started; log sequence number 0 44273
141212 15:54:33 [Note] Event Scheduler: Loaded 0 events
141212 15:54:33 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.67' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution
150119 8:06:38 [Note] /usr/libexec/mysqld: Normal shutdown
150119 8:06:38 [Note] Event Scheduler: Purging the queue. 0 events
150119 8:06:40 InnoDB: Starting shutdown...
150119 8:06:44 InnoDB: Shutdown completed; log sequence number 0 44283
150119 8:06:44 [Note] /usr/libexec/mysqld: Shutdown complete
150119 08:06:44 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
150119 08:06:44 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
150119 8:06:44 InnoDB: Initializing buffer pool, size = 8.0M
150119 8:06:44 InnoDB: Completed initialization of buffer pool
150119 8:06:44 InnoDB: Started; log sequence number 0 44283
150119 8:06:44 [Note] Event Scheduler: Loaded 0 events
150119 8:06:44 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.67' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution
150320 14:23:45 [Note] /usr/libexec/mysqld: Normal shutdown
150320 14:23:45 [Note] Event Scheduler: Purging the queue. 0 events
150320 14:23:47 InnoDB: Starting shutdown...
150320 14:23:51 InnoDB: Shutdown completed; log sequence number 0 44293
150320 14:23:51 [Note] /usr/libexec/mysqld: Shutdown complete
150320 14:23:51 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
150320 14:23:52 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
150320 14:23:52 InnoDB: Initializing buffer pool, size = 8.0M
150320 14:23:52 InnoDB: Completed initialization of buffer pool
150320 14:23:52 InnoDB: Started; log sequence number 0 44293
150320 14:23:52 [Note] Event Scheduler: Loaded 0 events
150320 14:23:52 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.67' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution
150408 14:36:44 [Note] /usr/libexec/mysqld: Normal shutdown
150408 14:36:44 [Note] Event Scheduler: Purging the queue. 0 events
150408 14:36:46 InnoDB: Starting shutdown...
150408 14:36:49 InnoDB: Shutdown completed; log sequence number 0 44303
150408 14:36:49 [Note] /usr/libexec/mysqld: Shutdown complete
150408 14:36:49 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
150408 14:36:49 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
150408 14:36:49 InnoDB: Initializing buffer pool, size = 8.0M
150408 14:36:49 InnoDB: Completed initialization of buffer pool
150408 14:36:49 InnoDB: Started; log sequence number 0 44303
150408 14:36:49 [Note] Event Scheduler: Loaded 0 events
150408 14:36:49 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.67' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution
150409 18:46:43 [Note] /usr/libexec/mysqld: Normal shutdown
150409 18:46:43 [Note] Event Scheduler: Purging the queue. 0 events
150409 18:46:45 InnoDB: Starting shutdown...
150409 18:46:46 InnoDB: Shutdown completed; log sequence number 0 44303
150409 18:46:46 [Note] /usr/libexec/mysqld: Shutdown complete
150409 18:46:46 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
150409 18:47:00 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
150409 18:47:00 InnoDB: Initializing buffer pool, size = 8.0M
150409 18:47:00 InnoDB: Completed initialization of buffer pool
150409 18:47:00 InnoDB: Started; log sequence number 0 44303
150409 18:47:00 [Note] Event Scheduler: Loaded 0 events
150409 18:47:00 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.67' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution
[root@esu1l268 scripts]# service ndo2db start
Starting ndo2db: done.
[root@esu1l268 scripts]# tail -100 /var/log/messages
Apr 9 18:45:02 esu1l268 nagios: wproc: Core Worker 22208: job 5131 (pid=834): Dormant child reaped
Apr 9 18:45:02 esu1l268 nagios: wproc: Core Worker 22211: job 5131 (pid=890) timed out. Killing it
Apr 9 18:45:02 esu1l268 nagios: wproc: CHECK job 5131 from worker Core Worker 22211 timed out after 60.01s
Apr 9 18:45:02 esu1l268 nagios: wproc: host=esu1l318; service=check_FS_space_linux_by_sshpass;
Apr 9 18:45:02 esu1l268 nagios: wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
Apr 9 18:45:02 esu1l268 nagios: Warning: Check of service 'check_FS_space_linux_by_sshpass' on host 'esu1l 318' timed out after 60.007s!
Apr 9 18:45:02 esu1l268 nagios: wproc: Core Worker 22211: job 5131 (pid=890): Dormant child reaped
Apr 9 18:45:03 esu1l268 nagios: wproc: Core Worker 22245: job 5131 (pid=1150) timed out. Killing it
Apr 9 18:45:03 esu1l268 nagios: wproc: CHECK job 5131 from worker Core Worker 22245 timed out after 60.02s
Apr 9 18:45:03 esu1l268 nagios: wproc: host=esu1l321; service=check_FS_space_linux_by_sshpass;
Apr 9 18:45:03 esu1l268 nagios: wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
Apr 9 18:45:03 esu1l268 nagios: Warning: Check of service 'check_FS_space_linux_by_sshpass' on host 'esu1l 321' timed out after 60.017s!
Apr 9 18:45:03 esu1l268 nagios: wproc: Core Worker 22245: job 5131 (pid=1150): Dormant child reaped
Apr 9 18:45:04 esu1l268 nagios: wproc: Core Worker 22200: job 5132 (pid=1245) timed out. Killing it
Apr 9 18:45:04 esu1l268 nagios: wproc: CHECK job 5132 from worker Core Worker 22200 timed out after 60.01s
Apr 9 18:45:04 esu1l268 nagios: wproc: host=esu1l325; service=check_FS_space_linux_by_sshpass;
Apr 9 18:45:04 esu1l268 nagios: wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
Apr 9 18:45:04 esu1l268 nagios: Warning: Check of service 'check_FS_space_linux_by_sshpass' on host 'esu1l 325' timed out after 60.007s!
Apr 9 18:45:04 esu1l268 nagios: wproc: Core Worker 22200: job 5132 (pid=1245): Dormant child reaped
Apr 9 18:45:07 esu1l268 nagios: wproc: Core Worker 22207: job 5133 (pid=2386) timed out. Killing it
Apr 9 18:45:07 esu1l268 nagios: wproc: CHECK job 5133 from worker Core Worker 22207 timed out after 60.01s
Apr 9 18:45:07 esu1l268 nagios: wproc: host=esu1l327; service=check_FS_space_linux_by_sshpass;
Apr 9 18:45:07 esu1l268 nagios: wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
Apr 9 18:45:07 esu1l268 nagios: Warning: Check of service 'check_FS_space_linux_by_sshpass' on host 'esu1l 327' timed out after 60.006s!
Apr 9 18:45:07 esu1l268 nagios: wproc: Core Worker 22207: job 5133 (pid=2386): Dormant child reaped
Apr 9 18:45:09 esu1l268 nagios: wproc: Core Worker 22244: job 5133 (pid=2900) timed out. Killing it
Apr 9 18:45:09 esu1l268 nagios: wproc: CHECK job 5133 from worker Core Worker 22244 timed out after 60.01s
Apr 9 18:45:09 esu1l268 nagios: wproc: host=esu1l326; service=check_FS_space_linux_by_sshpass;
Apr 9 18:45:09 esu1l268 nagios: wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
Apr 9 18:45:09 esu1l268 nagios: Warning: Check of service 'check_FS_space_linux_by_sshpass' on host 'esu1l 326' timed out after 60.006s!
Apr 9 18:45:09 esu1l268 nagios: wproc: Core Worker 22244: job 5133 (pid=2900): Dormant child reaped
Apr 9 18:45:13 esu1l268 nagios: wproc: Core Worker 22208: job 5135 (pid=3757) timed out. Killing it
Apr 9 18:45:13 esu1l268 nagios: wproc: CHECK job 5135 from worker Core Worker 22208 timed out after 60.01s
Apr 9 18:45:13 esu1l268 nagios: wproc: host=esu2v262; service=Check for read only File system on linux;
Apr 9 18:45:13 esu1l268 nagios: wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
Apr 9 18:45:13 esu1l268 nagios: Warning: Check of service 'Check for read only File system on linux' on ho st 'esu2v262' timed out after 60.005s!
Apr 9 18:45:13 esu1l268 nagios: wproc: Core Worker 22208: job 5135 (pid=3757): Dormant child reaped
Apr 9 18:45:16 esu1l268 vasd[2315]: _create_rule_from_local_file: Cannot locate <FEDERATED.FDS\UxRG_esu1l2 68>(UxRG_esu1l268) from file </etc/opt/quest/vas/users.allow>, error: <2>. Resolution will be attempted aga in in 30 seconds.
Apr 9 18:45:25 esu1l268 nagios: wproc: Core Worker 22248: job 5140 (pid=7581) timed out. Killing it
Apr 9 18:45:25 esu1l268 nagios: wproc: CHECK job 5140 from worker Core Worker 22248 timed out after 60.01s
Apr 9 18:45:25 esu1l268 nagios: wproc: host=scj3dbadm01; service=check_FS_space_Solaris_by_sshpass;
Apr 9 18:45:25 esu1l268 nagios: wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
Apr 9 18:45:25 esu1l268 nagios: Warning: Check of service 'check_FS_space_Solaris_by_sshpass' on host 'scj 3dbadm01' timed out after 60.006s!
Apr 9 18:45:25 esu1l268 nagios: wproc: Core Worker 22248: job 5140 (pid=7581): Dormant child reaped
Apr 9 18:45:38 esu1l268 nagios: wproc: Core Worker 22219: job 5145 (pid=11115) timed out. Killing it
Apr 9 18:45:38 esu1l268 nagios: wproc: CHECK job 5145 from worker Core Worker 22219 timed out after 60.01s
Apr 9 18:45:38 esu1l268 nagios: wproc: host=esu2v660; service=Check for read only File system on linux;
Apr 9 18:45:38 esu1l268 nagios: wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
Apr 9 18:45:38 esu1l268 nagios: Warning: Check of service 'Check for read only File system on linux' on ho st 'esu2v660' timed out after 60.006s!
Apr 9 18:45:38 esu1l268 nagios: wproc: Core Worker 22219: job 5145 (pid=11115): Dormant child reaped
Apr 9 18:45:39 esu1l268 nagios: wproc: Core Worker 22237: job 5145 (pid=11234) timed out. Killing it
Apr 9 18:45:39 esu1l268 nagios: wproc: CHECK job 5145 from worker Core Worker 22237 timed out after 60.01s
Apr 9 18:45:39 esu1l268 nagios: wproc: host=esu2v219; service=Check for read only File system on linux;
Apr 9 18:45:39 esu1l268 nagios: wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
Apr 9 18:45:39 esu1l268 nagios: Warning: Check of service 'Check for read only File system on linux' on ho st 'esu2v219' timed out after 60.008s!
Apr 9 18:45:39 esu1l268 nagios: wproc: Core Worker 22237: job 5145 (pid=11234): Dormant child reaped
Apr 9 18:45:41 esu1l268 nagios: wproc: Core Worker 22205: job 5146 (pid=11753) timed out. Killing it
Apr 9 18:45:41 esu1l268 nagios: wproc: CHECK job 5146 from worker Core Worker 22205 timed out after 60.01s
Apr 9 18:45:41 esu1l268 nagios: wproc: host=sun21; service=check_FS_space_Solaris_by_sshpass;
Apr 9 18:45:41 esu1l268 nagios: wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
Apr 9 18:45:41 esu1l268 nagios: Warning: Check of service 'check_FS_space_Solaris_by_sshpass' on host 'sun 21' timed out after 60.008s!
Apr 9 18:45:41 esu1l268 nagios: wproc: Core Worker 22205: job 5146 (pid=11753): Dormant child reaped
Apr 9 18:45:47 esu1l268 vasd[2315]: _create_rule_from_local_file: Cannot locate <FEDERATED.FDS\UxRG_esu1l2 68>(UxRG_esu1l268) from file </etc/opt/quest/vas/users.allow>, error: <2>. Resolution will be attempted aga in in 30 seconds.
Apr 9 18:45:51 esu1l268 nagios: wproc: Core Worker 22200: job 5150 (pid=13976) timed out. Killing it
Apr 9 18:45:51 esu1l268 nagios: wproc: CHECK job 5150 from worker Core Worker 22200 timed out after 60.01s
Apr 9 18:45:51 esu1l268 nagios: wproc: host=esu1l316; service=check_FS_space_linux_by_sshpass;
Apr 9 18:45:51 esu1l268 nagios: wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
Apr 9 18:45:51 esu1l268 nagios: Warning: Check of service 'check_FS_space_linux_by_sshpass' on host 'esu1l 316' timed out after 60.006s!
Apr 9 18:45:51 esu1l268 nagios: wproc: Core Worker 22200: job 5150 (pid=13976): Dormant child reaped
Apr 9 18:45:53 esu1l268 nagios: wproc: Core Worker 22234: job 5150 (pid=14403) timed out. Killing it
Apr 9 18:45:53 esu1l268 nagios: wproc: CHECK job 5150 from worker Core Worker 22234 timed out after 60.01s
Apr 9 18:45:53 esu1l268 nagios: wproc: host=esu1l044; service=check_FS_space_linux_by_sshpass;
Apr 9 18:45:53 esu1l268 nagios: wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
Apr 9 18:45:53 esu1l268 nagios: Warning: Check of service 'check_FS_space_linux_by_sshpass' on host 'esu1l 044' timed out after 60.007s!
Apr 9 18:45:53 esu1l268 nagios: wproc: Core Worker 22234: job 5150 (pid=14403): Dormant child reaped
Apr 9 18:45:55 esu1l268 nagios: wproc: Core Worker 22215: job 5151 (pid=14765) timed out. Killing it
Apr 9 18:45:55 esu1l268 nagios: wproc: CHECK job 5151 from worker Core Worker 22215 timed out after 60.01s
Apr 9 18:45:55 esu1l268 nagios: wproc: host=esu1l332; service=check_FS_space_linux_by_sshpass;
Apr 9 18:45:55 esu1l268 nagios: wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
Apr 9 18:45:55 esu1l268 nagios: Warning: Check of service 'check_FS_space_linux_by_sshpass' on host 'esu1l 332' timed out after 60.007s!
Apr 9 18:45:55 esu1l268 nagios: wproc: Core Worker 22215: job 5151 (pid=14765): Dormant child reaped
Apr 9 18:45:59 esu1l268 nagios: wproc: Core Worker 22236: job 5152 (pid=15707) timed out. Killing it
Apr 9 18:45:59 esu1l268 nagios: wproc: CHECK job 5152 from worker Core Worker 22236 timed out after 60.01s
Apr 9 18:45:59 esu1l268 nagios: wproc: host=esu3v128; service=Check for read only File system on linux;
Apr 9 18:45:59 esu1l268 nagios: wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
Apr 9 18:45:59 esu1l268 nagios: Warning: Check of service 'Check for read only File system on linux' on ho st 'esu3v128' timed out after 60.005s!
Apr 9 18:45:59 esu1l268 nagios: wproc: Core Worker 22236: job 5152 (pid=15707): Dormant child reaped
Apr 9 18:46:01 esu1l268 nagios: Caught SIGTERM, shutting down...
Apr 9 18:46:01 esu1l268 nagios: Successfully shutdown... (PID=22192)
Apr 9 18:46:01 esu1l268 nagios: Event broker module 'NERD' deinitialized successfully.
Apr 9 18:46:01 esu1l268 nagios: HP BSM Integration: Event Broker Module has been successfully unloaded.
Apr 9 18:46:01 esu1l268 nagios: Event broker module '/opt/OV/HPBsmIntNagios/lib64/libbsmintneb4.so' deinit ialized successfully.
Apr 9 18:46:01 esu1l268 nagios: ndomod: Shutdown complete.
Apr 9 18:46:01 esu1l268 nagios: Event broker module '/usr/local/nagios/bin/ndomod.o' deinitialized success fully.
Apr 9 18:46:17 esu1l268 vasd[2315]: _create_rule_from_local_file: Cannot locate <FEDERATED.FDS\UxRG_esu1l2 68>(UxRG_esu1l268) from file </etc/opt/quest/vas/users.allow>, error: <2>. Resolution will be attempted aga in in 30 seconds.
Apr 9 18:46:32 esu1l268 snmpd[2527]: Connection from UDP: [11.48.116.70]:48920->[11.48.4.85]
Apr 9 18:46:48 esu1l268 vasd[2315]: _create_rule_from_local_file: Cannot locate <FEDERATED.FDS\UxRG_esu1l2 68>(UxRG_esu1l268) from file </etc/opt/quest/vas/users.allow>, error: <2>. Resolution will be attempted aga in in 30 seconds.
Apr 9 18:47:19 esu1l268 vasd[2315]: _create_rule_from_local_file: Cannot locate <FEDERATED.FDS\UxRG_esu1l2 68>(UxRG_esu1l268) from file </etc/opt/quest/vas/users.allow>, error: <2>. Resolution will be attempted aga in in 30 seconds.
Apr 9 18:47:42 esu1l268 sshd[24527]: Accepted keyboard-interactive/pam for b161172 from 11.48.70.73 port 2 5197 ssh2
Apr 9 18:47:49 esu1l268 vasd[2315]: _create_rule_from_local_file: Cannot locate <FEDERATED.FDS\UxRG_esu1l2 68>(UxRG_esu1l268) from file </etc/opt/quest/vas/users.allow>, error: <2>. Resolution will be attempted aga in in 30 seconds.
[root@esu1l268 scripts]#
tail -100 /usr/local/nagios/var/nagios.log:
Code: Select all
[root@esu1l268 ~]# tail -100 /usr/local/nagios/var/nagios.log
[1428620223] wproc: host=esu2v805; service=Check for read only File system on linux;
[1428620223] wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
[1428620223] Warning: Check of service 'Check for read only File system on linux' on host 'esu2v805' timed out after 60.011s!
[1428620223] wproc: Core Worker 24762: job 139 (pid=7841): Dormant child reaped
[1428620224] wproc: Core Worker 24753: job 139 (pid=7879) timed out. Killing it
[1428620224] wproc: CHECK job 139 from worker Core Worker 24753 timed out after 60.01s
[1428620224] wproc: host=esu2v823; service=Check for read only File system on linux;
[1428620224] wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
[1428620224] Warning: Check of service 'Check for read only File system on linux' on host 'esu2v823' timed out after 60.010s!
[1428620224] wproc: Core Worker 24753: job 139 (pid=7879): Dormant child reaped
[1428620230] wproc: Core Worker 24752: job 142 (pid=8419) timed out. Killing it
[1428620230] wproc: CHECK job 142 from worker Core Worker 24752 timed out after 60.01s
[1428620230] wproc: host=esu2v974; service=Check for read only File system on linux;
[1428620230] wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
[1428620230] Warning: Check of service 'Check for read only File system on linux' on host 'esu2v974' timed out after 60.006s!
[1428620230] wproc: Core Worker 24752: job 142 (pid=8419): Dormant child reaped
[1428620233] wproc: Core Worker 24748: job 143 (pid=8602) timed out. Killing it
[1428620233] wproc: CHECK job 143 from worker Core Worker 24748 timed out after 60.01s
[1428620233] wproc: host=esu1l321; service=Check for read only File system on linux;
[1428620233] wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
[1428620233] Warning: Check of service 'Check for read only File system on linux' on host 'esu1l321' timed out after 60.013s!
[1428620233] wproc: Core Worker 24748: job 143 (pid=8602): Dormant child reaped
[1428620240] SERVICE ALERT: scj3-appadm01;Ping;CRITICAL;SOFT;1;CRITICAL - 11.48.37.140: rta nan, lost 100%
[1428620240] HP BSM Integration: Forwarding event for service 'Ping' (scj3-appadm01)...
[1428620242] wproc: Core Worker 24774: job 146 (pid=9496) timed out. Killing it
[1428620242] wproc: CHECK job 146 from worker Core Worker 24774 timed out after 60.01s
[1428620242] wproc: host=esu2v653; service=Check for read only File system on linux;
[1428620242] wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
[1428620242] Warning: Check of service 'Check for read only File system on linux' on host 'esu2v653' timed out after 60.008s!
[1428620242] wproc: Core Worker 24774: job 146 (pid=9496): Dormant child reaped
[1428620242] wproc: Core Worker 24779: job 146 (pid=9621) timed out. Killing it
[1428620242] wproc: CHECK job 146 from worker Core Worker 24779 timed out after 60.01s
[1428620242] wproc: host=esu4v110; service=Check for read only File system on linux;
[1428620242] wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
[1428620242] Warning: Check of service 'Check for read only File system on linux' on host 'esu4v110' timed out after 60.006s!
[1428620242] wproc: Core Worker 24779: job 146 (pid=9621): Dormant child reaped
[1428620242] wproc: Core Worker 24745: job 147 (pid=9743) timed out. Killing it
[1428620242] wproc: CHECK job 147 from worker Core Worker 24745 timed out after 60.01s
[1428620242] wproc: host=esu4v128; service=Check for read only File system on linux;
[1428620242] wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
[1428620242] Warning: Check of service 'Check for read only File system on linux' on host 'esu4v128' timed out after 60.007s!
[1428620242] wproc: Core Worker 24745: job 147 (pid=9743): Dormant child reaped
[1428620243] wproc: Core Worker 24755: job 147 (pid=9769) timed out. Killing it
[1428620243] wproc: CHECK job 147 from worker Core Worker 24755 timed out after 60.01s
[1428620243] wproc: host=esu4v133; service=Check for read only File system on linux;
[1428620243] wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
[1428620243] Warning: Check of service 'Check for read only File system on linux' on host 'esu4v133' timed out after 60.006s!
[1428620243] wproc: Core Worker 24755: job 147 (pid=9769): Dormant child reaped
[1428620246] wproc: Core Worker 24768: job 148 (pid=10422) timed out. Killing it
[1428620246] wproc: CHECK job 148 from worker Core Worker 24768 timed out after 60.01s
[1428620246] wproc: host=ibm104p01; service=check_FS_space_AIX_by_sshpass;
[1428620246] wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
[1428620246] Warning: Check of service 'check_FS_space_AIX_by_sshpass' on host 'ibm104p01' timed out after 60.008s!
[1428620246] wproc: Core Worker 24768: job 148 (pid=10422): Dormant child reaped
[1428620247] wproc: Core Worker 24774: job 148 (pid=10558) timed out. Killing it
[1428620247] wproc: CHECK job 148 from worker Core Worker 24774 timed out after 60.01s
[1428620247] wproc: host=ibm104p04; service=check_FS_space_AIX_by_sshpass;
[1428620247] wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
[1428620247] Warning: Check of service 'check_FS_space_AIX_by_sshpass' on host 'ibm104p04' timed out after 60.005s!
[1428620247] wproc: Core Worker 24774: job 148 (pid=10558): Dormant child reaped
[1428620250] HOST ALERT: scj3-appadm01;DOWN;SOFT;1;CRITICAL - 11.48.37.140: rta nan, lost 100%
[1428620250] HP BSM Integration: Forwarding event for host 'scj3-appadm01'...
[1428620250] wproc: Core Worker 24779: job 149 (pid=11077) timed out. Killing it
[1428620250] wproc: CHECK job 149 from worker Core Worker 24779 timed out after 60.01s
[1428620250] wproc: host=ibm70p7; service=check_FS_space_AIX_by_sshpass;
[1428620250] wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
[1428620250] Warning: Check of service 'check_FS_space_AIX_by_sshpass' on host 'ibm70p7' timed out after 60.010s!
[1428620250] wproc: Core Worker 24779: job 149 (pid=11077): Dormant child reaped
[1428620251] wproc: Core Worker 24747: job 150 (pid=11197) timed out. Killing it
[1428620251] wproc: CHECK job 150 from worker Core Worker 24747 timed out after 60.01s
[1428620251] wproc: host=ibm78p02; service=check_FS_space_AIX_by_sshpass;
[1428620251] wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
[1428620251] Warning: Check of service 'check_FS_space_AIX_by_sshpass' on host 'ibm78p02' timed out after 60.008s!
[1428620251] wproc: Core Worker 24747: job 150 (pid=11197): Dormant child reaped
[1428620251] wproc: Core Worker 24775: job 150 (pid=11204) timed out. Killing it
[1428620251] wproc: CHECK job 150 from worker Core Worker 24775 timed out after 60.01s
[1428620251] wproc: host=ibm78p11; service=check_FS_space_AIX_by_sshpass;
[1428620251] wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
[1428620251] Warning: Check of service 'check_FS_space_AIX_by_sshpass' on host 'ibm78p11' timed out after 60.007s!
[1428620251] wproc: Core Worker 24775: job 150 (pid=11204): Dormant child reaped
[1428620297] wproc: Core Worker 24766: job 159 (pid=21904) timed out. Killing it
[1428620297] wproc: CHECK job 159 from worker Core Worker 24766 timed out after 60.01s
[1428620297] wproc: host=scr1db02; service=check_FS_space_Solaris_by_sshpass;
[1428620297] wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
[1428620297] Warning: Check of service 'check_FS_space_Solaris_by_sshpass' on host 'scr1db02' timed out after 60.009s!
[1428620297] wproc: Core Worker 24766: job 159 (pid=21904): Dormant child reaped
[1428620297] SERVICE ALERT: scj3-appadm01;Ping;CRITICAL;SOFT;2;CRITICAL - 11.48.37.140: rta nan, lost 100%
[1428620307] HOST ALERT: scj3-appadm01;DOWN;SOFT;2;CRITICAL - 11.48.37.140: rta nan, lost 100%
[1428620316] wproc: Core Worker 24758: job 162 (pid=26873) timed out. Killing it
[1428620316] wproc: CHECK job 162 from worker Core Worker 24758 timed out after 60.01s
[1428620316] wproc: host=ibmpci00p02; service=check_FS_space_AIX_by_sshpass;
[1428620316] wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
[1428620316] Warning: Check of service 'check_FS_space_AIX_by_sshpass' on host 'ibmpci00p02' timed out after 60.011s!
[1428620316] wproc: Core Worker 24758: job 162 (pid=26873): Dormant child reaped
[1428620321] wproc: Core Worker 24776: job 162 (pid=27858) timed out. Killing it
[1428620321] wproc: CHECK job 162 from worker Core Worker 24776 timed out after 60.01s
[1428620321] wproc: host=esu2v221; service=Check for read only File system on linux;
[1428620321] wproc: early_timeout=1; exited_ok=0; wait_status=0; error_code=62;
[1428620321] Warning: Check of service 'Check for read only File system on linux' on host 'esu2v221' timed out after 60.015s!
[1428620321] wproc: Core Worker 24776: job 162 (pid=27858): Dormant child reaped
[root@esu1l268 ~]#
Re: Nagios XI Configuration Random Configuration Error
Posted: Fri Apr 10, 2015 8:07 am
by JakeHatMacys
And just curious do you guys ever do any remote sessions? Would that be a viable option here? (I know we recently upgraded our support package to include Phone Support) While I was thinking about just rolling back to a good configuration snapshot I'd really like to find out what has this hosed up.
We were supposed to go live on this server Monday. If this were to happen while we were live in production it'd be a huge problem for us.
And to be clear Core is still running, so I'm assuming we're still functioning but XI is hosed:
Capture.JPG
Re: Nagios XI Configuration Random Configuration Error
Posted: Fri Apr 10, 2015 10:12 am
by tmcdonald
We certainly do perform remote sessions, but we usually only do those once a ticket has been opened (I do not recall anyone having done a remote for a forum thread). If you want to move this to a ticket, please email
[email protected] and someone will be assigned to the ticket. Make sure to provide a descriptive email subject, and link to this thread in the body of the email.
If that is not something you want to do, please reply to this thread indicating such and we will continue here.