Page 1 of 2
Hosts and services disappears for a while
Posted: Thu May 16, 2019 9:23 am
by ITOMB_IMT
HI,
Recently we upgraded NagiosXI to 5.6.1 hosted on RHEL 7.5 and currently we are facing issues with CCM where Hosts and services disappear for a while and after running reconfigure script/multiple refreshes the Hosts/services appear. This happens every time when we make changes to Hosts/services in CCM. This is annoying to not see the hosts/services and running the script all the time.
Re: Hosts and services disappears for a while
Posted: Thu May 16, 2019 10:39 am
by benjaminsmith
Hi
@ISSB_MAOST,
You might have crashed database tables, please try to run the database repair script by running the following command as root:
Code: Select all
/usr/local/nagiosxi/scripts/repair_databases.sh
Then restart entire Nagios XI stack:
Code: Select all
systemctl stop crond
systemctl stop npcd
systemctl stop nagios
systemctl stop ndo2db
pkill -9 -u nagios
for i in $(ipcs -q | grep nagios |awk '{print $2}'); do ipcrm -q $i; done
rm -rf /usr/local/nagiosxi/var/dbmaint.lock
rm -rf /usr/local/nagiosxi/var/event_handler.lock
rm -rf /usr/local/nagiosxi/scripts/reconfigure_nagios.lock
systemctl restart mariadb
systemctl start ndo2db
systemctl start nagios
systemctl start npcd
systemctl start crond
Let me know if the issue is corrected or not.
Re: Hosts and services disappears for a while
Posted: Thu May 16, 2019 11:00 am
by ITOMB_IMT
No, still issue exists.
If i make any change and apply configuration, then the hosts/services disappear. Later i have to run re-configure.sh script, then the hosts/services appears again.
Re: Hosts and services disappears for a while
Posted: Thu May 16, 2019 11:13 am
by benjaminsmith
Hi
@ISSB_MAOST,
Let's tail the log file while you apply configuration to see what might be the source of the issue. Run the following command:
Code: Select all
tail -F /usr/local/nagiosxi/var/cmdsubsys.log
and then apply configuration and post/upload the entire output.
Next, please send us your system profile to review. Thanks.
To send us your system profile.
Login to the Nagios XI GUI using a web browser.
Click the "Admin" > "System Profile" Menu
Click the "Download Profile" button
Save the profile.zip file and share in a private message or upload it to the post/ticket.
Re: Hosts and services disappears for a while
Posted: Thu May 16, 2019 11:49 am
by ITOMB_IMT
I sent you a Pm with the cmdsubsys.log, when i try to download system profile am seeing the following error.
PROFILE BUILD FAILED
Array
(
)
CODE: 1
Re: Hosts and services disappears for a while
Posted: Thu May 16, 2019 1:05 pm
by benjaminsmith
Hi
@ISSB_MAOST,
That error is likely the result of an incorrect sodoers file, so we'll want to fix that first and then test again. Please follow the steps in the knowledge base article below to correct.
Nagios XI - Profile Build Failed
Re: Hosts and services disappears for a while
Posted: Thu May 16, 2019 1:28 pm
by ITOMB_IMT
I followed all the steps specified, instead of 5.5.9 i tried with 5.6.1 but still am seeing the same error. I have sent you a pm of our sudoers file, please have a look.
Thanks,
Re: Hosts and services disappears for a while
Posted: Thu May 16, 2019 3:45 pm
by benjaminsmith
Hi
@ISSB_MAOST,
I took a look the sudoers file and it does look ok, so I'd like to check the following.
1. Do you have SE Linux enabled?
2. Is the Nagios User account expired?
If it is expired, run this command to enable the expired nagios user:
Code: Select all
chage -I -1 -m 0 -M 99999 -E -1 nagios
I noticed this in the command subsystem log:
touch: cannot touch ‘/usr/local/nagiosxi/scripts/reconfigure_nagios.lock’: Permission denied
Please PM the following file so we can review the permissions.
Code: Select all
ls -lR /usr/local/nagios/ > /tmp/support.txt
Thanks.
Re: Hosts and services disappears for a while
Posted: Thu May 16, 2019 3:55 pm
by ITOMB_IMT
Please see the output of the commnads
# getenforce
Disabled
# chage -l nagios
Last password change : Oct 16, 2018
Password expires : never
Password inactive : never
Account expires : never
Minimum number of days between password change : -1
Maximum number of days between password change : -1
Number of days of warning before password expires : -1
and also see the PM of the output of ls -lR /usr/local/nagios/ > /tmp/support.txt
Re: Hosts and services disappears for a while
Posted: Thu May 16, 2019 4:54 pm
by benjaminsmith
Hi
@ISSB_MAOST,
Well, it looks like there are multiple issues present. One is the inability to generate a system profile, and issue with hosts and services disappearing is likely a database issue.
1. Can you run the following tail command to get the Apache error logs:
.. and then try to download the system profile from Admin > System Config > System Profile. Please PM the error output from the tail command.
2. What version of mysql are you using?
3. Can you post the output of the following database query and the db logs?
A. Login
B. Change database, and run the query
Code: Select all
use nagiosql;
SELECT @@SQL_MODE, @@GLOBAL.SQL_MODE\G;
The database logs are in
/var/log/mysqld.log
Thanks.