Page 1 of 3

all host pending

Posted: Tue Sep 26, 2017 11:16 pm
by sangmin
Hi,

There was a problem with nagios xi. Only one host appears in the Host Status Summary as normal.
The normal list of hosts is changed randomly.

If you look at the status of the host in the service status, it is all pending.

I tried restarting the server, restarting nagios, restarting the DB, but the results are the same.

Re: all host pending

Posted: Wed Sep 27, 2017 9:11 am
by tgriep
Lets check and see if Nagios Core shows all of the Hosts, run the following on the Nagios server as root and post the output.

Code: Select all

/usr/local/nagios/bin/nagiostats
ipcs -q
ps -ef --cols=300
df -h
df -i
Nagios XI displays the status from the MYSQL database and maybe the table is corrupted so it is not displaying the data.
Run the following to repair the database and see if that fixes the issue.

Code: Select all

cd /usr/local/nagiosxi/scripts
./repair_databases.sh
Let us know the outcome.

Re: all host pending

Posted: Wed Sep 27, 2017 9:19 am
by lmiltchev
Having hosts in a pending state could be caused by a number of things - not having a check command assigned to hosts, disabled active checks, DB issues, etc. In order for us to narrow it down, I would recommend that you start troubleshooting by following the steps, outlined in our "Nagios XI - Hosts Pending/Gray" KB article:

https://support.nagios.com/kb/article/n ... -gray.html

Please check out all of the possible causes, listed in the article above.

If you are still having an issue, PM me or anyone on the Nagios tech team your profile.

Admin > System Profile > Download Profile

Re: all host pending

Posted: Wed Sep 27, 2017 8:31 pm
by sangmin
In the Nagios core, the host is output normally.
I ran ./repair_databases.sh, but the issue was not resolved.
db was offloaded from mysql.

I also included the nagios profile and the
"/usr/local/nagios/bin/nagiostats
ipcs -q
ps -ef --cols=300
df -h
df -i"
result in the archive.

Re: all host pending

Posted: Thu Sep 28, 2017 12:31 pm
by lmiltchev
I see lots of errors in the apache error log, similar to this one:

Code: Select all

Can't connect to MySQL server on xxx
If you have DB errors/crashed tables, you will need to repair them on the offloaded server.

You can run:

Code: Select all

mysqlcheck -r -f -uroot -p<password> --all-databases
where you substitute <password> with the actual mysql root password.

Have you tried restarting mysqld on the offloaded server?

Did you by chance change the default mysql port? Let's see if mysqld is running and if there are mysql errors in the log. Please run the following commands on the offloaded mysql server, and show the output in code wraps:

Code: Select all

ps -ef | grep mysql | grep -v grep
tail -50 /var/log/mysqld.log
Can you access mysql from the Nagios XI server? Try:

Code: Select all

echo 'SELECT NOW();' | mysql -t -uroot -p<password> -h <ip address>

Re: all host pending

Posted: Sat Sep 30, 2017 8:20 pm
by sangmin
mysqlcheck does not seem to work properly.
I installed mysql manually.
It uses innodb and differs from the default setting.
The mysql connection seems to be blocked by default. Access is good when connecting as nagios user.

Attach the results.

thanks.

Re: all host pending

Posted: Mon Oct 02, 2017 10:09 am
by scottwilkerson
This is going to be a problem, none of the tables in the nagios database should have a db engine of innodb

Someone in your organization must have changed the DB engine.

Re: all host pending

Posted: Tue Oct 03, 2017 7:17 am
by sangmin
Can I fix this by changing the mysql engine to myisam? If so, I will change the engine.

Re: all host pending

Posted: Tue Oct 03, 2017 9:18 am
by scottwilkerson
sangmin wrote:Can I fix this by changing the mysql engine to myisam? If so, I will change the engine.
I'd say yes, however if the tables are corrupt you may not be able to change the engine.

It is worth an attempt, if it fails you will likely have to restore the database from a backup anyways.

If you can change the tables successfully, you should be able to run the database repair script

Re: all host pending

Posted: Tue Oct 10, 2017 2:04 am
by sangmin
Hi,

I changed the db engine to myisam and I succeeded in running the repair_databases script. But the result is the same. As before, only one host is visible.

Attach the result.

Thanks.