Page 1 of 1
No hosts or services showing in CCM & additional error
Posted: Tue May 19, 2020 6:21 am
by caf_infra
Nagiosxi version 5.5.2, running on Centos 7
Hi,
We've got a problem with our Nagios Xi installation. The CCM isn't showing any objects at all, though the system is still correctly monitoring our hosts and services.
When I tried the trouble shooting steps in another post (Delete - Write - Verify -Restart) with a similar issue, I get the error "Cannot find the Nagios binary or no rights for execution!". I have checked the perms on the /usr/local/nagios/bin/nagios file, and ran chmod +x against it (perms are now 775, file ownership is nagios:nagios)
None of the file systems on the nagios system are full.
Any advice please?
Re: No hosts or services showing in CCM & additional error
Posted: Tue May 19, 2020 11:48 am
by caf_infra
Hi,
Would really like some advice, because we also need to reboot the nagios server for other reasons, and I want to be confident that it'll work once it's been rebooted.
Thanks
Re: No hosts or services showing in CCM & additional error
Posted: Tue May 19, 2020 2:16 pm
by jbrunkow
You might just want to start by trying to repair the database by running the script below.
Code: Select all
/usr/local/nagiosxi/scripts/repair_databases.sh
If that doesn't work, send us a profile and we can examine that to determine what might be going awry. You can download the profile by navigating to
System Profile under
System Config on the left pane > and clicking the
Download Profile button on that page. Then please reply to this topic to bring it up in our queue.
Re: No hosts or services showing in CCM & additional error
Posted: Wed May 20, 2020 7:28 am
by caf_infra
HI,
I've tried that, no change. I'm not sure it is a database issue, because it's continuing to monitor everything atm.
I've tried the reset_config_perms.sh script too because some of the errors we're getting imply a perms issue.
What's the best way to get the profile to you? Can we email it rather than uploading to the forum?
Re: No hosts or services showing in CCM & additional error
Posted: Wed May 20, 2020 10:49 am
by jbrunkow
Dang! It is fairly common for issues like these to be related to the database. Thanks for trying it.
Yes, of course! You could send it to me in a private message by clicking the PM icon next to one of my comments.
Re: No hosts or services showing in CCM & additional error
Posted: Wed May 20, 2020 11:47 am
by caf_infra
Sent, thank you.
Re: No hosts or services showing in CCM & additional error
Posted: Wed May 20, 2020 4:40 pm
by jbrunkow
It looks like the
database log in your profile is indicating a problem finding your table settings.
Code: Select all
[ERROR] mysqld: Can't find file: './nagiosql/tbl_settings.frm' (errno: 117)
Can you please look to see if there is anything in that folder on your server?
This error could also indicate a permissions issue with that folder or the files in it. If you run the following command, the first column in the output will show you your permissions to that file.
If you have a backup, it may be easier to just revert
XI to the last working state it was in. The only drawback is that you would lose performance data gathered since that backup was made. Please refer to the following linked documentation on how to do so.
BACKUP DOCUMENT
Re: No hosts or services showing in CCM & additional error
Posted: Thu May 21, 2020 6:50 am
by caf_infra
Everything is coming up in Red and Black (seems to indicate a broken file according to the bash documentation) in the CLI and with 'Structure Needs Cleaning' Errors. I can't see any sign of that file anywhere, not even a broken remnant.
As I said in my PM, we've got the same issue with files showing as red/black and with that 'Structure needs cleaning' error in other parts of the installation.
Re: No hosts or services showing in CCM & additional error
Posted: Thu May 21, 2020 2:06 pm
by jbrunkow
Do you have a backup for that table that you can roll back to? You can check with the following command.
As for the missing
tbl_settings.frm file, you might be able to create it using the following command.
Code: Select all
echo 'repair table tbl_settings use_frm;' | mysql -h 127.0.0.1 -uroot -pnagiosxi nagiosql
Then check to see if that worked.
Several sources online are reporting that
xfs_repair and
e2fsck resolved similar file corruption errors. I recommend that you perform a backup of the damaged file systems, and then try to repair them using one of the two utilities. One or the other may already be present on your machine depending on the operating system...