all host pending

[sangmin]

Hi,

There was a problem with nagios xi. Only one host appears in the Host Status Summary as normal.
The normal list of hosts is changed randomly.

If you look at the status of the host in the service status, it is all pending.

I tried restarting the server, restarting nagios, restarting the DB, but the results are the same.

[tgriep]

Lets check and see if Nagios Core shows all of the Hosts, run the following on the Nagios server as root and post the output.

Code: Select all

/usr/local/nagios/bin/nagiostats
ipcs -q
ps -ef --cols=300
df -h
df -i

Nagios XI displays the status from the MYSQL database and maybe the table is corrupted so it is not displaying the data.
Run the following to repair the database and see if that fixes the issue.

Code: Select all

cd /usr/local/nagiosxi/scripts
./repair_databases.sh

Let us know the outcome.

[lmiltchev]

Having hosts in a pending state could be caused by a number of things - not having a check command assigned to hosts, disabled active checks, DB issues, etc. In order for us to narrow it down, I would recommend that you start troubleshooting by following the steps, outlined in our "Nagios XI - Hosts Pending/Gray" KB article:

https://support.nagios.com/kb/article/n ... -gray.html

Please check out all of the possible causes, listed in the article above.

If you are still having an issue, PM me or anyone on the Nagios tech team your profile.

Admin > System Profile > Download Profile

[sangmin]

In the Nagios core, the host is output normally.
I ran ./repair_databases.sh, but the issue was not resolved.
db was offloaded from mysql.

I also included the nagios profile and the
"/usr/local/nagios/bin/nagiostats
ipcs -q
ps -ef --cols=300
df -h
df -i"
result in the archive.

[lmiltchev]

I see lots of errors in the apache error log, similar to this one:

Code: Select all

Can't connect to MySQL server on xxx

If you have DB errors/crashed tables, you will need to repair them on the offloaded server.

You can run:

Code: Select all

mysqlcheck -r -f -uroot -p<password> --all-databases

where you substitute <password> with the actual mysql root password.

Have you tried restarting mysqld on the offloaded server?

Did you by chance change the default mysql port? Let's see if mysqld is running and if there are mysql errors in the log. Please run the following commands on the offloaded mysql server, and show the output in code wraps:

Code: Select all

ps -ef | grep mysql | grep -v grep
tail -50 /var/log/mysqld.log

Can you access mysql from the Nagios XI server? Try:

Code: Select all

echo 'SELECT NOW();' | mysql -t -uroot -p<password> -h <ip address>

[sangmin]

mysqlcheck does not seem to work properly.
I installed mysql manually.
It uses innodb and differs from the default setting.
The mysql connection seems to be blocked by default. Access is good when connecting as nagios user.

Attach the results.

thanks.

[scottwilkerson]

This is going to be a problem, none of the tables in the nagios database should have a db engine of innodb

Someone in your organization must have changed the DB engine.

[sangmin]

Can I fix this by changing the mysql engine to myisam? If so, I will change the engine.

[scottwilkerson]

Can I fix this by changing the mysql engine to myisam? If so, I will change the engine.

I'd say yes, however if the tables are corrupt you may not be able to change the engine.

It is worth an attempt, if it fails you will likely have to restore the database from a backup anyways.

If you can change the tables successfully, you should be able to run the database repair script

[sangmin]

Hi,

I changed the db engine to myisam and I succeeded in running the repair_databases script. But the result is the same. As before, only one host is visible.

Attach the result.

Thanks.