Page 2 of 4
Re: Database Connection Error
Posted: Tue Feb 17, 2015 12:17 pm
by grenley
tail -f shows that no new messages have been written to the mysql log since yesterday (probably about the time I ran the repair_databases script):
Code: Select all
[u@h W]$ tail -f /opt/app/mysql/log/mysqld.log
150216 20:23:14 InnoDB: Completed initialization of buffer pool
150216 20:23:14 InnoDB: highest supported file format is Barracuda.
150216 20:23:14 InnoDB: Waiting for the background threads to start
150216 20:23:15 InnoDB: 1.1.8 started; log sequence number 1595675
150216 20:23:15 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
150216 20:23:15 [Note] - '0.0.0.0' resolves to '0.0.0.0';
150216 20:23:15 [Note] Server socket created on IP: '0.0.0.0'.
150216 20:23:15 [Note] Event Scheduler: Loaded 0 events
150216 20:23:15 [Note] //opt/app/mysql/bin/mysqld: ready for connections.
Version: '5.5.25-log' socket: '//opt/app/mysql/mysql.sock' port: 3306 MySQL Community Server (GPL)
Despite that, we continue to receive the messages to run that command and CCM says there are no hosts (though it does show services).
Could it be that the hosts table is corrupted?
Is there a way to reload it?
Re: Database Connection Error
Posted: Tue Feb 17, 2015 12:29 pm
by abrist
grenley wrote:Could it be that the hosts table is corrupted?
Possibly, but you should get an error that the repair was unsuccessful. Try to repair the hosts table with:
Code: Select all
echo "repair table nagios.nagios_hosts;" | mysql -pnagiosxi
grenley wrote:Is there a way to reload it?
Yes, though you could just truncate it and let ndo refill the table.
Re: Database Connection Error
Posted: Tue Feb 17, 2015 1:02 pm
by grenley
Okay. The repair came back with an OK status:
Code: Select all
[u@h W]$ echo "repair table nagios.nagios_hosts;" | mysql -u root -p******** nagios --socket=/opt/app/mysql/mysql.sock
Table Op Msg_type Msg_text
nagios.nagios_hosts repair status OK
But CCM still shows "Error connecting to database." when I select Hosts under the Monitoring section.
It says "No results returned from host table"
When I select Services, a list of services
is displayed (along with "Error connecting to database.").
It also asks me to apply a new configuration (which is odd since we haven't used CCM recently to manage the configuration).
As soon as I click "Apply Configuration", I get the following...
Code: Select all
An error occurred while attempting to apply your configuration to Nagios Core. Monitoring engine configuration files have been rolled back to their last known good checkpoint.
But they aren't really rolled back as it just recurs when I repeat the process.
Re: Database Connection Error
Posted: Tue Feb 17, 2015 1:42 pm
by lmiltchev
Run the following commands and show us the output in code wraps:
Code: Select all
service nagios restart
service nagios status
service ndo2db restart
service ndo2db status
service mysqld restart
service mysqld status
tail -20 /var/log/mysqld.log
grep ndo /var/log/messages | tail -20
Re: Database Connection Error
Posted: Wed Feb 18, 2015 11:17 am
by grenley
It appears our installation process (not actually owned by the Nagios team) does not install these as services.
Here's what we get when I run the commands you provided:
Code: Select all
nagios: unrecognized service
nagios: unrecognized service
ndo2db: unrecognized service
ndo2db: unrecognized service
mysqld: unrecognized service
mysqld: unrecognized service
Here's what we use to stop these processes:
Code: Select all
su - nagios -c "/usr/local/nagios/etc/ndoutils/init.d/ndo2db-lrm-stop"
su - mysql -c "/opt/app/platform/init.d/mysql.server stop"
su - nagios -c "/usr/local/nagios/etc/init.d/nagios-lrm-stop"
(plus the similar start commands)
Will this suffice for your purposes?
Re: Database Connection Error
Posted: Wed Feb 18, 2015 11:26 am
by abrist
They will indeed.
Re: Database Connection Error
Posted: Wed Feb 18, 2015 11:37 am
by grenley
Code: Select all
[rg8239@zltc5114 ~]$ tail -20 /opt/app/mysql/log/mysqld.log
150218 16:32:30 mysqld_safe mysqld from pid file //opt/app/mysql/data/zltc5114.pid ended
150218 16:32:54 mysqld_safe Starting mysqld daemon with databases from //opt/app/mysql/data
150218 16:32:54 [Warning] Changed limits: max_open_files: 1024 max_connections: 214 table_cache: 400
150218 16:32:55 [Note] Plugin 'FEDERATED' is disabled.
150218 16:32:55 InnoDB: The InnoDB memory heap is disabled
150218 16:32:55 InnoDB: Mutexes and rw_locks use GCC atomic builtins
150218 16:32:55 InnoDB: Compressed tables use zlib 1.2.3
150218 16:32:55 InnoDB: Using Linux native AIO
150218 16:32:55 InnoDB: Initializing buffer pool, size = 128.0M
150218 16:32:55 InnoDB: Completed initialization of buffer pool
150218 16:32:55 InnoDB: highest supported file format is Barracuda.
150218 16:32:55 InnoDB: Waiting for the background threads to start
150218 16:32:56 InnoDB: 1.1.8 started; log sequence number 1595675
150218 16:32:56 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
150218 16:32:56 [Note] - '0.0.0.0' resolves to '0.0.0.0';
150218 16:32:56 [Note] Server socket created on IP: '0.0.0.0'.
150218 16:32:56 [Note] Event Scheduler: Loaded 0 events
150218 16:32:56 [Note] //opt/app/mysql/bin/mysqld: ready for connections.
Version: '5.5.25-log' socket: '//opt/app/mysql/mysql.sock' port: 3306 MySQL Community Server (GPL)
There are no ndo messages in /var/log/messages
Re: Database Connection Error
Posted: Wed Feb 18, 2015 12:55 pm
by abrist
Is it possible to move this too a ticket (for a profile.zip) and then possibly to a remote session? If so, send an email to
[email protected] and lets resolve this.
Re: Database Connection Error
Posted: Wed Feb 18, 2015 4:46 pm
by grenley
sure. happy to do it.
what do you mean by profile.zip?
Re: Database Connection Error
Posted: Wed Feb 18, 2015 4:50 pm
by abrist
In XI, go to Admin --> System profile --> download. Send the profile.zip that you downloaded with your email to
[email protected]. Also, include a link to this thread.
Thanks!