Page 1 of 1
Unable to start Monitoring Engine
Posted: Wed Aug 21, 2013 5:43 pm
by rks
My NagiosXI installation was rebooted. After reboot, I find that the
monitoring service and performance grapher are no longer running (red exclamation mark).
When I attempt to start it manually (off the GUI gear icon), it yields a message that
"the command could not be processed in a timely manner").
Pl help!
rk
Re: Unable to start Monitoring Engine
Posted: Wed Aug 21, 2013 5:45 pm
by rks
Btw, I see some ominous messages in the log:
Code: Select all
Aug 21 18:41:38 ctvmanager ndo2db: mysql_error: 'Table './nagios/nagios_systemcommands' is marked as crashed and should be repaired'
Aug 21 18:41:38 ctvmanager ndo2db: Error: mysql_query() failed for 'INSERT INTO nagios_systemcommands SET instance_id='1', start_time=FROM_UNIXTIME(1377124898), start_time_usec='126691', end_time=FROM_UNIXTIME(1377124898), end_time_usec='130434', command_line='/bin/mv /usr/local/nagios/var/service-perfdata /usr/local/nagios/var/spool/xidpe/1377124898\.perfdata\.service', timeout='5', early_timeout='0', execution_time='0.003000', return_code='0', output='', long_output='' ON DUPLICATE KEY UPDATE instance_id='1', start_time=FROM_UNIXTIME(1377124898), start_time_usec='126691', end_time=FROM_UNIXTIME(1377124898), end_time_usec='130434', command_line='/bin/mv /usr/local/nagios/var/service-perfdata /usr/local/nagios/var/spool/xidpe/1377124898\.perfdata\.service', timeout='5', early_timeout='0', execution_time='0.003000', return_code='0', output='', long_output='''
Aug 21 18:41:38 ctvmanager ndo2db: mysql_error: 'Table './nagios/nagios_systemcommands' is marked as crashed and should be repaired'
It appears that the MySQL database is encountering errors. Could you pl suggest a root cause?
thanks,
rk
Re: Unable to start Monitoring Engine
Posted: Wed Aug 21, 2013 10:39 pm
by scottwilkerson
Run the following
Code: Select all
/usr/local/nagiosxi/scripts/repairmysql.sh nagios
Re: Unable to start Monitoring Engine
Posted: Thu Aug 22, 2013 11:08 am
by rks
Thanks. I rant the script and it seemingly executed successfully:
However, I did not stop services (was I supposed to?).
Code: Select all
[root@ctvmanager ~]# /usr/local/nagiosxi/scripts/repairmysql.sh nagios
DATABASE: nagios
TABLE:
/var/lib/mysql/nagios ~
Stopping mysqld: [ OK ]
- recovering (with sort) MyISAM-table 'nagios_acknowledgements.MYI'
Data records: 2
- Fixing index 1
---------
// .....
- recovering (with sort) MyISAM-table 'nagios_timeperiod_timeranges.MYI'
Data records: 33
- Fixing index 1
- Fixing index 2
Starting mysqld: [ OK ]
~
===============
REPAIR COMPLETE
===============
[root@ctvmanager ~]#
After the script completed, I see error messages in the log about connections failures to mysql:
Code: Select all
Aug 22 12:00:16 ctvmanager ndo2db: Error: Could not connect to MySQL database: Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
Aug 22 12:00:16 ctvmanager ndo2db: Error: Could not connect to MySQL database: Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
Aug 22 12:00:16 ctvmanager ndo2db: Error: Could not connect to MySQL database: Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
Aug 22
And I am still not able to start either the monitoring engine or the performance grapher.
thanks!
rk
Re: Unable to start Monitoring Engine
Posted: Thu Aug 22, 2013 11:25 am
by lmiltchev
Let's see what services do you have running. Run the following commands, and show the output:
Code: Select all
service nagios status
service ndo2db status
service mysqld status
service postgresql status
service crond status
service npcd status
Also, run the following commands, and show the output:
Code: Select all
service mysqld restart
tail /var/log/mysqld.log
Re: Unable to start Monitoring Engine
Posted: Thu Aug 22, 2013 2:41 pm
by rks
Thanks! crond had stopped. I restarted it (and mysqld as well) and everything is green.
Code: Select all
[root@ctvmanager ~]# service nagios status
nagios (pid 29192) is running...
[root@ctvmanager ~]# service ndo2db status
ndo2db (pid 1819) is running...
[root@ctvmanager ~]# service mysqld status
mysqld (pid 15539) is running...
[root@ctvmanager ~]# service postgresql status
postmaster (pid 1452) is running...
[root@ctvmanager ~]# service crond status
crond (pid 5863) is running...
[root@ctvmanager ~]# service npcd status
NPCD running (pid 1639).
[root@ctvmanager ~]# service mysqld restart
Stopping mysqld: [ OK ]
Starting mysqld: [ OK ]
[root@ctvmanager ~]# tail /var/log/mysqld.log
130822 15:36:28 [Note] /usr/libexec/mysqld: Shutdown complete
130822 15:36:28 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
130822 15:36:29 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
130822 15:36:29 InnoDB: Initializing buffer pool, size = 8.0M
130822 15:36:29 InnoDB: Completed initialization of buffer pool
130822 15:36:29 InnoDB: Started; log sequence number 0 44233
130822 15:36:29 [Note] Event Scheduler: Loaded 0 events
130822 15:36:29 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.67' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution
[root@ctvmanager ~]# tail -f /var/log/mysqld.log
130822 15:36:28 [Note] /usr/libexec/mysqld: Shutdown complete
130822 15:36:28 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
130822 15:36:29 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
130822 15:36:29 InnoDB: Initializing buffer pool, size = 8.0M
130822 15:36:29 InnoDB: Completed initialization of buffer pool
130822 15:36:29 InnoDB: Started; log sequence number 0 44233
130822 15:36:29 [Note] Event Scheduler: Loaded 0 events
130822 15:36:29 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.67' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution
Thanks!
rk