Tom,
I checked the /var/log/mysqld.log => No logs since 23/04
These are the last entries:
Code: Select all
150413 14:01:35 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
150413 14:02:52 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
150413 14:02:53 InnoDB: Initializing buffer pool, size = 8.0M
150413 14:02:53 InnoDB: Completed initialization of buffer pool
150413 14:02:53 InnoDB: Started; log sequence number 0 44233
150413 14:02:53 [Note] Event Scheduler: Loaded 0 events
150413 14:02:53 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.73' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution
150422 10:14:54 [Note] /usr/libexec/mysqld: Normal shutdown
150422 10:14:54 [Note] Event Scheduler: Purging the queue. 0 events
150422 10:14:55 InnoDB: Starting shutdown...
150422 10:15:00 InnoDB: Shutdown completed; log sequence number 0 44233
150422 10:15:00 [Note] /usr/libexec/mysqld: Shutdown complete
150422 10:15:00 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
150422 10:17:24 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
150422 10:17:25 InnoDB: Initializing buffer pool, size = 8.0M
150422 10:17:25 InnoDB: Completed initialization of buffer pool
150422 10:17:25 InnoDB: Started; log sequence number 0 44233
150422 10:17:26 [Note] Event Scheduler: Loaded 0 events
150422 10:17:26 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.73' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution
150422 10:52:31 [Note] /usr/libexec/mysqld: Normal shutdown
150422 10:52:31 [Note] Event Scheduler: Purging the queue. 0 events
150422 10:52:33 InnoDB: Starting shutdown...
150422 10:52:38 InnoDB: Shutdown completed; log sequence number 0 44233
150422 10:52:38 [Note] /usr/libexec/mysqld: Shutdown complete
150422 10:52:38 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
150422 10:54:11 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
150422 10:54:12 InnoDB: Initializing buffer pool, size = 8.0M
150422 10:54:12 InnoDB: Completed initialization of buffer pool
150422 10:54:12 InnoDB: Started; log sequence number 0 44233
150422 10:54:12 [Note] Event Scheduler: Loaded 0 events
150422 10:54:12 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.73' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution
150423 14:38:55 [Note] /usr/libexec/mysqld: Normal shutdown
150423 14:38:55 [Note] Event Scheduler: Purging the queue. 0 events
150423 14:38:56 InnoDB: Starting shutdown...
150423 14:38:57 InnoDB: Shutdown completed; log sequence number 0 44233
150423 14:38:57 [Note] /usr/libexec/mysqld: Shutdown complete
150423 14:38:57 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
150423 14:49:24 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
150423 14:49:25 InnoDB: Initializing buffer pool, size = 8.0M
150423 14:49:25 InnoDB: Completed initialization of buffer pool
150423 14:49:25 InnoDB: Started; log sequence number 0 44233
150423 14:49:25 [Note] Event Scheduler: Loaded 0 events
150423 14:49:25 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.73' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution
I tried creating the dig_srv_win_drv_A_usage_prio2 service template and then renamed a host. Same problem. Applying configuration takes a very long time. THis is the logging from cmdsubsys
Code: Select all
tail -f /usr/local/nagiosxi/var/cmdsubsys.log
...........................................................
PROCESSED 0 COMMANDS
tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated
................................................PROCESSING COMMAND ID 18119...
PROCESS COMMAND: CMD=17, DATA=
APPLYING NAGIOSCORE CONFIG...
CMDLINE=cd /usr/local/nagiosxi/scripts && ./reconfigure_nagios.sh
URL: http://localhost/nagiosxi/includes/components/ccm/
CMDLINE
--2015-06-04 09:00:50-- http://localhost/nagiosxi/includes/components/ccm/
Resolving localhost... ::1, 127.0.0.1
Connecting to localhost|::1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: “nagiosql.login”
0K .......... . 934K=0.01s
2015-06-04 09:00:50 (934 KB/s) - “nagiosql.login” saved [11589]
/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies http://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'submit=Login&hidelog=true&loginSubmitted=true&username=nagiosxi&password=pasw' -O nagiosql.loginLOGIN SUCCESSFUL!
IMPORTING CONFIG FILES...URL: http://localhost/nagiosxi/includes/components/ccm/
Array
(
)
RESETTING PERMS
SETUID ROOT OK
URL: http://localhost/nagiosxi/includes/components/ccm/
CMDLINE
--2015-06-04 09:00:51-- http://localhost/nagiosxi/includes/components/ccm/
Resolving localhost... ::1, 127.0.0.1
Connecting to localhost|::1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: “nagiosql.login”
0K .......... . 1.01M=0.01s
2015-06-04 09:00:51 (1.01 MB/s) - “nagiosql.login” saved [11589]
/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies http://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'submit=Login&hidelog=true&loginSubmitted=true&username=nagiosxi&password=pasw' -O nagiosql.loginLOGIN SUCCESSFUL!
URL: http://localhost/nagiosxi/includes/components/ccm/
CMDLINE:
/usr/bin/wget --load-cookies=nagiosql.cookies http://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'cmd=apply&type=writeConfig' -O nagiosql.export.monitoring
--2015-06-04 09:00:51-- http://localhost/nagiosxi/includes/components/ccm/
Resolving localhost... ::1, 127.0.0.1
Connecting to localhost|::1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: “nagiosql.export.monitoring”
0K ........tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated
............................................................tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated
.
PROCESSED 0 COMMANDS
tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated
PROCESSED 0 COMMANDS
. 74.4 =2m15s
2015-06-04 09:03:07 (74.4 B/s) - “nagiosql.export.monitoring” saved [10078]
NAGIOSQL WRITE CONFIGS FAILED!\n
OUTPUT=NAGIOSQL WRITE CONFIGS FAILED!\n
RETURNCODE=5
PROCESSED 1 COMMANDS
tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated
............................................................tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated
.
PROCESSED 0 COMMANDS
tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated
.
PROCESSED 0 COMMANDS
In the gui, I still get this error: Error: Template 'dig_srv_win_drv_A_usage_prio2' specified in service definition could not be not found (config file '/usr/local/nagios/etc/services/testserver.cfg', starting on line 52)
For the first time I did not try to revert to the last ok config cnapshot and tried to apply one more time. tailing cmdsubsys.log again gave
Code: Select all
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
So I guess this implies the apply config after renaming a host actually takes even longer. And it seems the config actually gets applied the second time (after a long time)!
Code: Select all
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
Another reconfigure process is still running, sleeping...
RESETTING PERMS
SETUID ROOT OK
URL: http://localhost/nagiosxi/includes/components/ccm/
CMDLINE
--2015-06-04 09:12:50-- http://localhost/nagiosxi/includes/components/ccm/
Resolving localhost... ::1, 127.0.0.1
Connecting to localhost|::1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: “nagiosql.login”
0K .......... . 1.07M=0.01s
2015-06-04 09:12:50 (1.07 MB/s) - “nagiosql.login” saved [11589]
/usr/bin/wget --save-cookies nagiosql.cookies --keep-session-cookies http://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'submit=Login&hidelog=true&loginSubmitted=true&username=nagiosxi&password=passw' -O nagiosql.loginLOGIN SUCCESSFUL!
URL: http://localhost/nagiosxi/includes/components/ccm/
CMDLINE:
/usr/bin/wget --load-cookies=nagiosql.cookies http://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'cmd=apply&type=writeConfig' -O nagiosql.export.monitoring
--2015-06-04 09:12:51-- http://localhost/nagiosxi/includes/components/ccm/
Resolving localhost... ::1, 127.0.0.1
Connecting to localhost|::1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: “nagiosql.export.monitoring”
0K ........tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated
PROCESSED 0 COMMANDS
.. .......... . 1.38K=15s
2015-06-04 09:13:06 (1.38 KB/s) - “nagiosql.export.monitoring” saved [21688]
WRITE CONFIGS SUCCESSFUL!
OUTPUT:
Nagios Core 4.0.8
Copyright (c) 2009-present Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 08-12-2014
License: GPL
Website: http://www.nagios.org
Reading configuration data...
Read main config file okay...
Read object config files okay...
Running pre-flight check on configuration data...
Checking objects...
Checked 12689 services.
Checked 744 hosts.
Checked 114 host groups.
Checked 83 service groups.
Checked 65 contacts.
Checked 16 contact groups.
Checked 284 commands.
Checked 79 time periods.
Checked 0 host escalations.
Checked 0 service escalations.
Checking for circular paths...
Checked 744 hosts
Checked 0 service dependencies
Checked 0 host dependencies
Checked 79 timeperiods
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...
Total Warnings: 0
Total Errors: 0
Things look okay - No serious problems were detected during the pre-flight check
RET: 0
Running configuration check...
Stopping nagios:. done.
Starting nagios: done.
OUTPUT=Starting nagios: done.
RETURNCODE=0
PROCESSED 1 COMMANDS
tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated
............................................................tail: /usr/local/nagiosxi/var/cmdsubsys.log: file truncated
.
PROCESSED 0 COMMANDS
And the host was correctly renamed. The weird thing when looking at config changes is that a whole bunch of hosts that have nothing to do with the renamed hosts have also been rewritten.. I don't understand what's going on here.
EDIT:
SO I decided to give my XI prod server a reboot and did all the above tests again. It seems things are better now. Renaming a host actually works the first apply, but it still takes a very long time. Looking at the config changes of the apply show he wrote all the config files of all my 700 hosts, so I guess that's why it takes so long. This does not seem 'normal' to me? Or is it by design that after renaming one hosts, all hosts are rewritten? I would suspent only the affected host and service cfg (of the renamed host) should be written?
Grtz
